Python Forum
embedding Python script in shell scripts
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
embedding Python script in shell scripts
#1
i'm starting to think through how to embed a Python script inside a shell script. the first issue that comes to mind is keep the data input and code input separate. i don't want to write the code to a separate file because it may be that there is no file system mounted R/W. everything could be R/O. if i could tell Python to read the code from a specific file descriptor, that might work.

this involves Python code too big to squeeze onto one line. it might be on the next 100 lines of the shell script. in Linux i could use something like /proc/self/fd/9 as the source code to run an input lines below via that descriptor, like 9<<EOF and put EOF after the last line of code. if this works, then i would have each piece of Python code be organized as a shell function so it can be used like a command in a long shell script pipeline. the idea is that i can get started converting just critical pieces of shell scripts to Python without have to wait until i have time to convert the whole thing to Python.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
It could be easier to embed shell scripts into a python script. Did you consider this option?
Reply
#3
You can do this with here-doc.

#!/bin/bash

echo "Hello World from bash"

python <<EOF
from __future__ import print_function
import sys
print(sys.version)
print("Using a variable from shell: $PWD")
EOF

python <<'EOF'
from __future__ import print_function
print('Without variables: $PWD')
EOF

echo "Preparing a list in shell"
li=(1 2 3 4 5 6)
echo ${li[@]}

python <<EOF
from __future__ import print_function
print('Using list from shell in Pyhon')
numbers = list(map(int, "${li[@]}".split()))
print(numbers)
EOF

if [ -x '/usr/bin/python3' ]; then
/usr/bin/python3 <<EOF
import sys
print('Python 3 exists on this system....')
print(sys.version)
EOF
fi


if [ -x '/usr/bin/python3' ]; then
        PYTHON='/usr/bin/python3'
elif [ -x '/usr/bin/python2' ]; then
        PYTHON='/usr/bin/python2'
elif [ -x '/usr/bin/python' ]; then
        PYTHON='/usr/bin/python'
fi

$PYTHON <<'EOF'
from __future__ import print_function
import sys
print('Using detected Python interpreter...')
print(sys.version)
EOF
Output:
Hello World from bash 2.7.14 (default, Sep 23 2017, 22:06:14) [GCC 7.2.0] Using a variable from shell: /home/andre Without variables: $PWD Preparing a list in shell 1 2 3 4 5 6 Using list from shell in Pyhon [1, 2, 3, 4, 5, 6] Python 3 exists on this system.... 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] Using detected Python interpreter... 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0]
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#4
the "here-doc" approach is what i used. i never heard it called that (i called it "embedded input"). i never participated in a bash forum ... all they had at the time were mailing lists and i don't like mailing lists ... so that may explain never seeing that terminology. i am glad there is a forum around for Python. anyway, it worked just fine.

i had previously organized this (originally a "one off") script with a bunch of functions so i could keep lines relatively short (max 184, my text screen width) and separate pipelines. the functions had "awk" commands that were not working. i rewrote them 3 times and gave up and switched to Python. the first coding in Python worked. now i am thinking of reducin these 3 functions to just 1 to simplify the pipelines that call them.

in theory, i could have done it all in Python. but i already had a bash script to gather the data (a bunch of ssh and rsync calls) and at first i thought this would be a quickie mod. it wasn't.

as for which way to embed is best, i'd say just do the whole thing in Python. but in some cases, especially with pre-existing code, it can be better to make parts in another language. in the case of a part being in Python, it could be a separate Python script. but for many things, i like to keep things all in on file if it can remain readable. for Python in bash, i was hoping it would be simple enough to remain readable. i was certain it could be made to work. just how simple was my big question. fortunately the "here-doc" approach was quite simple.

#!/bin/bash
doit() {
    python3 /proc/self/fd/4 4<<EOF
print('this is python')
EOF
}

echo foo
doit
doit
echo bar
exit 0
i didn't need to use the -c option, which i thought would make things less readable.

if a command line option like "+4" could be added to mean: read the code from the specified file descriptor, this would have been even simpler.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#5
(Mar-22-2018, 01:47 AM)Skaperen Wrote: if a command line option like "+4" could be added to mean: read the code from the specified file descriptor, this would have been even simpler.
It seems that you can use /dev/fd/4 instead of /proc/self/fd/4
Reply
#6
(Mar-22-2018, 07:14 AM)Gribouillis Wrote:
(Mar-22-2018, 01:47 AM)Skaperen Wrote: if a command line option like "+4" could be added to mean: read the code from the specified file descriptor, this would have been even simpler.
It seems that you can use /dev/fd/4 instead of /proc/self/fd/4

on some distros, the /dev/fd link isn't there.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  embedded Python scripts - is it worthwhile Skaperen 3 510 Feb-21-2024, 01:25 AM
Last Post: Skaperen
  images and pixels in typical Python scripts Skaperen 0 1,194 Oct-14-2021, 12:29 AM
Last Post: Skaperen
  embedding data in a script Skaperen 1 2,038 May-31-2021, 07:19 AM
Last Post: Gribouillis
  Creating Python scripts in Visual Studio Code Sanjish 4 2,535 Dec-22-2020, 12:32 PM
Last Post: snippsat
  embedding python Skaperen 0 1,438 Apr-22-2020, 03:19 AM
Last Post: Skaperen
  suid scripts in python? Skaperen 0 2,113 Sep-29-2019, 10:28 PM
Last Post: Skaperen
  Python Shell As Preferred Mode For Integrated Terminal In Visual Studio Code adt 4 3,976 Sep-07-2019, 06:00 PM
Last Post: adt
  Can copyright be applied python scripts? tim777 3 4,791 Sep-04-2019, 12:20 PM
Last Post: DeaD_EyE
  Finding Scripts for Python Johnny1998 1 2,168 Jul-31-2019, 08:07 PM
Last Post: Yoriz
  Notepad vs anaconda spyder or python shell ShadowAlpha 2 2,916 Jun-26-2019, 06:31 AM
Last Post: perfringo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020