Python Forum
Thread Rating:
  • 2 Vote(s) - 4 Average
  • 1
  • 2
  • 3
  • 4
  • 5
pipeline between 2 programs
#11
Sounds similar to before. Using stdout=PIPE, you can just read from the process's .stdout file-like attribute. Then just keep looping through the processes until all of the processes .poll() method returns True.
Reply
#12
here's what i have put together so far:
def run_pipe_line(*a):
    from subprocess import PIPE, Popen
    if len(a) == 1 and isinstance(a[0],(list,tuple)):
        # a list of commands instead of one per arg
        a = a[0]
    l = len(a)
    if l < 1: # if no commands then no results
        return []
    if l < 2: # if just one command then just do it
        p = [Popen(a[0])]
    else:
        for n in range(l):
            if n < 1: # if this is first command
                p = [Popen(a[0],stdout=PIPE)]
            elif n == l-1: # if this is last command
                p.append(Popen(a[l-1],stdin=p[n-1].stdout))
            else:
                p.append(Popen(a[n],stdin=p[n-1].stdout,stdout=PIPE))
    r = []
    for n in range(l): # wait for all and collect results
        r.append(p[n].wait())
    return r
note that it can take either one command per argument or a list of commands as one argument. i like to be flexible like that. i did a little test of it a few minutes ago and it worked both ways, after fixing one bug on line 16 (the 'l-1' was 'l-2').

one reason i wanted to make this function (after many which i did not do it) is a program i wrote to do time stamps. it would prefix lines read from stdin and print them to stdout. it would also take arguments as a line to print with a time stamp. and it would also accept the "-e" option (or an "e" in the command name alias) to use the arguments as a command to execute and prefix that command's stdout with time stamps on each line. but this was implemented in a hackish way using bash and i wanted to make a cleaner pipeline. but now i realize even that is a bad hack because it runs a 2nd instance of itself. it should just read the executed command in to itself.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  builing a long variable-length command-pipeline Skaperen 6 4,362 Dec-14-2017, 02:02 AM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020