Python Forum
Capturing inputs values from internal python script - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Capturing inputs values from internal python script (/thread-19169.html)

Pages: 1 2


Capturing inputs values from internal python script - limors11 - Jun-16-2019

Hi there Rolleyes ,

I am using python 3.6 version. I have main python script which calls other python scripts, those internal scripts prompts for several user inputs. I wish to run those and then send the user input to the parent process and also send the result.
Note that the internal script has a lot of output, I don't want to take it all, but only the user input. Till now I wrote this info to a file and read it in the parent. But I am searching for a better elegant way which will allow me using any script as a black box and capturing its input and result without changing the inside script).
(At first I used os.system, check_output, call, popen also used PIPE, but could not capture it in a way the internal script continues, being completed and return all input to the user, it has several inputs in each script, till now it froze after the first or did not print the output, did not understand why).
I will appreciate if you can add code example of this and explains how can I implement that.

Please advice,
Thank you Smile ,
Limor


RE: Capturing inputs values from internal python script - noisefloor - Jun-16-2019

Hi,

Quote: have main python script which calls other python scripts,
Howdo you do you that? Importing the other scripts and call their functions, classes, ...? Or using subprocess or something similar? In case the latter applies, you do it wrong 99.9% for sure. The former is the way to go.

Except this, without seeing your code, the whole thing is vague. But to me it sounds like the overall design of the scripts need some refractoring, e.g. be returning results and not printing them to STDOUT.

As said, show your code (at least the relevant sections) and you'll get better help.

Regards, noisefloor


RE: Capturing inputs values from internal python script - limors11 - Jun-16-2019

Hi,

First thanks for your feedback.
I don't import other scripts or use the classes, those are black box separated scripts.
Till now I used methods like subprocess.getoutput(cmd) or check_output(..)
as the following:
try:
		data = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT) 
		exitCode = 0
except subprocess.CalledProcessError as ex:
		data = ex.output
		data2 = ex.cmd
		exitCode = ex.returncode
		writeToAll(logfile, "Command:\n " + str(cmd) + "\nResult:\n Error occurred: " + str(data) + " exit code: " + str(exitCode) + "(0 for SUCCESS). Build will exit.", loggingLevel )
		writeInfoToLog( logfile, "---------------------------------------------------------------------------------------------")
		if exitOnError:
			exit(exitCode)
	if data[-1:] == '\n':
		data = data[:-1]
	return data, exitCode
But this won't serve me now in others (I am just a python beginner, not an expert). I will add an example to explain my problem.
#child.py
#!/usr/bin/env python

import sys
# 
print('Enter user:')
user = input()
print('Enter password:' )
passwd = input()
print('Enter tag:')
tag = input()
#Parent.py
...
p = subprocess.Popen([PYTHON, "-i", "./child.py"], stdout=subprocess.PIPE, stdin=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
...
Don't know how to continue from here to pass the input parameters user inserted in child.py.

Hope this explains it.
Please advice,
Thank you for your help,
Limor


RE: Capturing inputs values from internal python script - noisefloor - Jun-16-2019

Hi,

please put your code into code blocks. You get them clicking on the Python logo button on the editor and enter your code between the [python]...[/python] tags. Without formating, the code is not readable.

Quote:those are black box separated scripts.
It was my understanding of your original post that the script(s) you call are Python scripts? Which means they hardly can be blackboxed.

Regards, noisefloor


RE: Capturing inputs values from internal python script - DeaD_EyE - Jun-16-2019

Please edit your post and do this: https://python-forum.io/misc.php?action=help&hid=25

Hint: The method communicate of Popen waits until the process has been finished.
So if you want to read data, when the process is running, you could not use communicate.

parent.py
import sys
from subprocess import (
    Popen, run, PIPE
    )

# running child process and wait until it has been finished
# this call blocks

print('Run the process and wait until the process finishes')
proc = run([sys.executable, 'child.py'], text=True, capture_output=True)
print(proc.stdout)

print('Run the process and do not wait until it has been finished')
# need to run Python in unbuffered mode, otherwise the data is buffered
# and you get the same result as before
proc = Popen([sys.executable, '-u' ,'child.py'], encoding='utf8', stdout=PIPE)

# proc.stdout is in TextMode and you can iterate over the lines
# you can do this with all files which are open in text mode, which is
# the default of open()
for line in proc.stdout:
    print(line)
child.py:
import time

time.sleep(5)
print('Hello World')
time.sleep(5)
print('Quit child')
Maybe this helps a little bit to understand.
If somewhere a buffer is used for stdout, you may get all messages after the program ends.
You can change this behavior with the option switch -u.


RE: Capturing inputs values from internal python script - limors11 - Jun-16-2019

Hi,

First thanks for your feedback.
I don't import other scripts or use the classes, those are black box separated scripts.
Till now I used methods like subprocess.getoutput(cmd) or check_output(..)
as the following:
try:
data = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT) 
exitCode = 0
except subprocess.CalledProcessError as ex:
data = ex.output
data2 = ex.cmd
exitCode = ex.returncode
writeToAll(logfile, "Command:\n " + str(cmd) + "\nResult:\n Error occurred: " + str(data) + " exit code: " + str(exitCode) + "(0 for SUCCESS). Build will exit.", loggingLevel )
writeInfoToLog( logfile, "---------------------------------------------------------------------------------------------")
if exitOnError:
exit(exitCode)
if data[-1:] == '\n':
data = data[:-1]
return data, exitCode
But this won't serve me now in others (I am just a python beginner, not an expert). I will add an example to explain my problem.
#child.py
#!/usr/bin/env python

import sys
# 
print('Enter user:')
user = input()
print('Enter password:' )
passwd = input()
print('Enter tag:')
tag = input()
#Parent.py
...
p = subprocess.Popen([PYTHON, "-i", "./child.py"], stdout=subprocess.PIPE, stdin=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
...
Don't know how to continue from here to pass the input parameters user inserted in child.py and received it in the parent.py and print it for example.

Hope this explains it.
Please advice,
Thank you for your help,
Limor


RE: Capturing inputs values from internal python script - noisefloor - Jun-16-2019

Hi,

Quote: Don't know how to continue from here to pass the input parameters user inserted in child.py and received it in the parent.py and print it for example.
Actually you don't do it like this - at least not for 2 Python scripts, as proceeding like this is super wrong. The way to go is using Python's import mechanism, function calls and binding results to the return value of the function / method called.

Example:

child.py:
def get_user_data():
    user = input('User name: ')
    user_id = input('User id: ')
    return user, user_id
parent.py:
import child

user, user_id = child.get_user_data()
print('User {} has the id {}.'.format(user, user_id))
Run it:
Output:
noisefloor@t450: python3 parent.py User name: noisefloor User id: 666 User noisefloor has the id 666.
Using subprocess is typically only necessary when calling a none-Python program script.

Quote:I am just a python beginner, not an expert
In case you didn't do yet, it's a good idea to read through the Python tutorial at docs.python.org. It also explains how function calls, importing and all the other basics work.

Regards, noisefloor


RE: Capturing inputs values from internal python script - limors11 - Jun-16-2019

Thank you I know all of that (and implemented that), this is just a really primitive example. I know how to use this package function call. You don't seem to follow me really at all. I don't won't to add any child code to the parent (this is why I called it black box, I already know and use those stuff you mentioned and it does not fit here in what I want). Isn't there a way? please just answer to this question if you can. Thank you again


RE: Capturing inputs values from internal python script - DeaD_EyE - Jun-16-2019

I have read it too late. You want to send data to your child process?
If you want to send something to the child process, use stdin=PIPE

from subprocess import Popen, PIPE
proc = Popen(['less'], stdin=PIPE, encoding='utf8')
proc.stdin.write('Hello World in less')
proc.stdin.flush()
proc.stdin.close()
The programm less displays files and can read also data from standard input.
If you remove proc.stdin.flush() or proc.stdin.close(), the program won't displays it (buffered).

To do both:

from subprocess import Popen, PIPE
proc = Popen(['less'], stdin=PIPE, stdout=PIPE, encoding='utf8')
proc.stdin.write('Hello World in less from less')
proc.stdin.flush()
proc.stdin.close()
for line in proc.stdout:
    print(line)
I use this method to start a WebSocketServer for a Sensor. The other part is a WebApp, which can switch over to another sensor, which is another technology and delivers different data and has different abilities. My app is now more monolithic and saves much CPU. Before I had massive issues with WebSockets, because of the messaging behind and with 90 frames per seconds, you won't buffer.. After this concept change my code was much smaller. But this is not always the right way. I still dislike it start a python process from a python programm.


RE: Capturing inputs values from internal python script - noisefloor - Jun-16-2019

Hi,

@limors11: I'm confused. First you say you are a beginner, than you know you know all of that. Fist you ask a question, than you alter the question, as you don't seem to like the answers.

May you should restart and describe your _real_ problem in detail, without making up artificial examples.

But independent of the: calling on Python script from another by subprocess is in 99.9% all all cases broken-by-design. No matter which way you turn it.

Generally speaking, if you want two independent processes - which may be two Python scripts - to communicate with each other - use an appropriate method of interprocess communication. Which could be a simple file, a named pipe, a SQL database, a KV store llike Redis or similar.

Regards, noisefloor