Python Forum
cheating at running a pipeline
Thread Rating:
  • 1 Vote(s) - 1 Average
  • 1
  • 2
  • 3
  • 4
  • 5
cheating at running a pipeline
#1
i wanted to run a pipeline of programs. this time i cheated and let the bash command do it. but there are problems passing command names and arguments reliably. there might be meta characters like space or dollar sign. i can quote them but then i could have quotes in quotes in the string passed to bash. single quotes don't support any kind of escaping in bash, so if a string has a single quote i have to use double quotes and deal with all the hassle those bring (escape everything). the way i did it was putting everything in numbered environment variables and built a bash command string referencing them in a pipeline structure. double quotes are used, but resolving from variables does not need escaping in that string. some day, i will finish my python pipeline function. i am not providing input or capturing output. if i was, it would be more complicated.

    os.environ['V1']="command1"
    os.environ['V2']="argument1"
    os.environ['V3']="command2"
    os.environ['V4']="argument2"
    subprocess.call(['bash','-c','"$V1" "$V2"|"$V3" "$V4"])
of course, there are many ways to set this up. longer environment variable names were used in the real script i wrote, keeping the name prefix in a variable so it is coded in just one place.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  running a pipeline of commands Skaperen 9 3,251 Dec-22-2020, 01:31 AM
Last Post: Skaperen
  Open Source Software for Data Pipeline Testing keaton8630 0 3,136 Dec-05-2019, 05:50 PM
Last Post: keaton8630

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020