Python Forum

Hello All,
I am trying to launch a daemon using subprocessor.Popen(), the daemon runs for about 30 seconds until it reaches a steady state.  My question is how can I determine it has reached the steady state before continuing my python execution?  
The daemon stops printing to screen once it has reached a steady state.  
Any help is appreciated, I don't have to use Popen if there are other better options.
Thanks!

try:
     p = subprocess.Popen(exe_path)
     #wait until it reaches steady state
     do_stuff()
except subprocess.CalledProcessError as e:
    print(e)
finallly:
    p.kill()

You could read the stdout of the process, and when there's nothing left to read you know it's "ready"?

Here's some fake code I didn't test:

proc = subprocess.Popen(exe_path, stdout=subprocess.PIPE)
while proc.stdout.peek():
   output = proc.stdout.read()
# no more output (or we read input before the process wrote anything) 

(Jul-07-2017, 04:07 PM)nilamo Wrote: [ -> ]You could read the stdout of the process, and when there's nothing left to read you know it's "ready"?

Here's some fake code I didn't test:

proc = subprocess.Popen(exe_path, stdout=subprocess.PIPE)
while proc.stdout.peek():
   output = proc.stdout.read()
# no more output (or we read input before the process wrote anything) 

Thanks, I tried this and it doesn't seem to exit the while loop, looks like .read() hangs waiting for a response when none is given (maybe waiting for EOF or something?). I tried 'continue' instead of .read() and it still hung. So maybe even .peek() is being problematic.

---

Sorry just realized .read() has to be used (or .readline()). Still hanging, its like it isn't finishing the .read()

All posted code is only for Python 3.

proc.stdout.read() blocks until EOF has been reached. This happens, when the process has been closed.
proc.stdout.readline() blocks until a new line is in the buffer.

It tested it with a shell script. I am not sure if it behaves like your daemon. It's different because the shell forks child processes.

daemon-sim.sh:

#!/bin/bash
sleep 4
echo "Running daemon..."
sleep 1
echo "Deamon is running now."
while true; do sleep 5; echo "`date`"; done

import subprocess
import re


def wait_for_daemon(path, args, ready_line_regex, shell=False):
   cmd = [path] + args
   proc = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE)
   while True:
       line = proc.stdout.readline() # returns bytes
       if ready_line_regex.search(line):
           break
   return proc


running_regex = re.compile(rb'Running daemon') # Using raw binary strting for regex
proc = wait_for_daemon('./daemon-sim.sh', [], running_regex, shell=True)
# don't use shell=True in your case, it's unsafe

I think this may solve your problem. But pay attention, if your daemon process is logging to stdout,
it's added to your memory. To prevent huge memory consumption, you should have a task, which is reading proc.stdout.
In most cases logging of daemons happens to stderr. In this case you're not redirecting stderr to somewhere. You'll see them in
your shell or if the daemon has a logging function, it logs to a file.

Good shell citizens are logging errors to stderr and data is written to stdout. A daemon process should not
log errors to stdout. But sometimes they do.

The solution above starts the process and redirect stdout to the PIPE.
Inside the while loop proc.stdout.readline() is called, which blocks until there is a new line.
When the check is True, the while loop is stopped (break) and the function returns proc.

A solution without regex:

import subprocess


def wait_for_daemon(path, args, ready_string, shell=False):
   cmd = [path] + args
   proc = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE)
   while True:
       line = proc.stdout.readline() # returns bytes
       if ready_string in line:
           break
   return proc


running_string = b'Running daemon' # Using binary string
proc = wait_for_daemon('./daemon-sim.sh', [], running_string, shell=True)
# don't use shell=True in your case, it's unsafe

You can also set encoding in Popen to 'utf-8' or other encoding, which depends on your daemon.
If you set the encoding, stdout.readline() returns a str and not bytes. If you don't set encoding, you'll receive bytes.