Problem Passing Arguement to do loop

[stephenmolnar]

I am running Debian Stretch on my main platform. I have a Computational Chemistry problem in which I have to run a number of python scripts. How do I pass the argument to a do loop?

I have a total of 32 ligands of the form name.mol2 that I want to dock to a number of protein molecules. The ligand is in a space separated ASCII file.

The computations will involve 17 different python scripts.

Here is my code:
!/bin/sh

for nvar in ligands do
/pythonsh ./prepare_ligand4.py -l $nvar.mol2 -o $nvar.pbdqt
done

When I run the code I get:

~/Apps/Modeling/VSTutorial/VirtualScreening/Ligands$ ./Run.pbdqt.sh setting PYTHONHOME environment the file ligands.mol2 doesn't contain any mol2 records The file ligands.mol2 doesn't have Atom records, molecules can't be built Traceback (most recent call last): File "./prepare_ligand4.py", line 152, in mol = mols[0] File "/sdc1/Apps/MGLTools-1.5.6rc3/lib/python2.5/UserList.py", line 28, in getitem def getitem(self, i): return self.data[i] IndexError: list index out of range

I know that the argument is not being passed, but have googled, but can't find any solution.

Assistance will be much appreciated.

Thanks in advance.

Stephen P. Molnar, Ph.D. Life is a fuzzy set
www.molecular-modeling.net Stochastic and multivariate
(614)312-7528©
Skype: smolnar1

[Gribouillis]

You could do it directly in python instead of using /bin/sh

#!/pythonsh
import subprocess
import sys
with open('ligand') as infile:
    ligand = infile.read().strip().split()
for nvar in ligand:
    subprocess.call([
        sys.executable, "./prepare_ligand4.py",
        "-l", nvar + ".mol2",
        "-o", nvar + ".pbdqt" ])

Alternately, if you're ready to change /bin/sh for /bin/bash, here is a way to delegate the work to python

#!/bin/bash

read -r -d '' somecode <<EOF
import subprocess
import sys
with open('ligand') as infile:
    ligand = infile.read().strip().split()
for nvar in ligand:
    subprocess.call([
        sys.executable, "./prepare_ligand4.py",
        "-l", nvar + ".mol2",
        "-o", nvar + ".pbdqt" ])
EOF

/pythonsh -c "$somecode"

I don't know how to do the same for /bin/sh . Everything is so much simpler in python!

[DeaD_EyE]

The first thing I did, was googling for the word ligands. Learned again...

Do you try to change the Pythonhome? If then, why?
Without knowledge about your scripts and the directory tree, it's like guessing.


A little trick you can do, is to use the main-script you start, as kind of anchor.

import sys
from pathlib import Path

# absolute path to your main program
# sys.argv[0] is the script file
# in most cases, this is a relative path
# absolute converts it for you
program_path = Path(sys.argv[0]).absolute()
program_dir = program_path.parent

# if your data is in program_dir/mol_data/

data_dir = program_dir / 'mol_data'

# globbing, get all mol-files
for mol_file in data_dir.glob('*.mol'):
    print(mol_file)
    # code to do something with the file

The resulting paths are all absolute, because the program_path which was used, is absolute.
No more problems with relative paths.

Just don't try to solve your problem in Bash. It's quick and dirty, but with Python you have a cleaner language and even an abstraction for Paths. This works on Linux, Mac and Windows.

[stephenmolnar]

I have made a little bit of progress with this code:
#!/pythonsh
import subprocess
import sys
with open('ligands') as infile:
ligand = infile.read().strip().split()
for nvar in ligand:
subprocess.call([
sys.executable, "./prepare_ligand4.py",
"-l", nvar + ".mol2",
"-o", nvar + ".pbdqt" ])

execution goes without error until the for statement. Then there is a series errors, one for each .mol file. The problem is that pythonsh is not a bash file, but rather a bash shell file which sets paths to files in MGLTools which are written in python, albeit it happens to be python2.5. As far as I know the authors of MGLTools have to plans to upgrade the version of python they have used.

If I run ./pythonsh ./prepare_ligand4.py -l <name>.mol2 the .pdbqt file is generated and saved. So I know he code works, just not as I would prefer.

[Gribouillis]

What happens if you replace sys.executable by "./pythonsh" in the subprocess.call or by os.path.join(os.getcwd(), 'pythonsh')? Also can you post the error message for one of the files?

[stephenmolnar]

Thanks for the reply.

I replaced sys.executable with "./pythonsh" and the program and the program ended with code: 0. However, no file was written.

[Gribouillis]

However, no file was written.

Try to get some information about the subprocesses. Replace the call() with

for nvar in ligand:
    command = [
        "./pythonsh", "./prepare_ligand4.py",
        "-l", nvar + ".mol2",
        "-o", nvar + ".pbdqt" ]
    proc = subprocess.Popen(command, stdout=subprocess.PIPE, stdout=subprocess.PIPE)
    out, err = proc.communicate()
    print(out)
    print(err)
    if proc.returncode:
        print 'Subprocess FAILED:', proc.command

This code should be compatible with python 2.5's Popen API.

[stephenmolnar]

Had to make some changes to your changes:

#!/usr/bin/env python2.5
# -*- coding: utf-8 -*-
import subprocess
import sys
with open('ligands') as infile:
ligand = infile.read().strip().split()
for nvar in ligand:
command = [
"./pythonsh", "./prepare_ligand4.py",
"-l", nvar + ".mol2",
"-o", nvar + ".pbdqt" ]
proc = subprocess.Popen(command, stdout=subprocess.PIPE)
out, err = proc.communicate()
print(out)
print(err)
if proc.returncode:
print ('Subprocess FAILED:', proc.command)

Here are the results:

runfile('/sdc1/Apps/Modeling/FileEditing/untitled1m2.py', wdir='/sdc1/Apps/Modeling/FileEditing')
Traceback (most recent call last):

File "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3296, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)

File "<ipython-input-1-51b00a34bbcd>", line 1, in <module>
runfile('/sdc1/Apps/Modeling/FileEditing/untitled1m2.py', wdir='/sdc1/Apps/Modeling/FileEditing')

File "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 824, in runfile
execfile(filename, namespace)

File "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "/sdc1/Apps/Modeling/FileEditing/untitled1m2.py", line 17
print 'Subprocess FAILED:', proc.command
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('Subprocess FAILED:', proc.command)?




runfile('/sdc1/Apps/Modeling/FileEditing/untitled1m2.py', wdir='/sdc1/Apps/Modeling/FileEditing')
Traceback (most recent call last):

File "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3296, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)

File "<ipython-input-2-51b00a34bbcd>", line 1, in <module>
runfile('/sdc1/Apps/Modeling/FileEditing/untitled1m2.py', wdir='/sdc1/Apps/Modeling/FileEditing')

File "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 824, in runfile
execfile(filename, namespace)

File "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "/sdc1/Apps/Modeling/FileEditing/untitled1m2.py", line 12
proc = subprocess.Popen(command, stdout=subprocess.PIPE, stdout=subprocess.PIPE)
^
SyntaxError: keyword argument repeated




runfile('/sdc1/Apps/Modeling/FileEditing/untitled1m2.py', wdir='/sdc1/Apps/Modeling/FileEditing')
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None
b''
None

[DeaD_EyE]

You're mixing Python 2.7 and 3.x code.
You could convert the scripts with 2to3 or other tools.
Another approach could be, to have two different installed Python-Versions.

Then you can't use sys.executable, if you run Python3.
In this case the sys.executable is Python3.

On the most Linux systems /usr/bin/python is a symlink to python2.7 and /usr/bin/python3 is a symlink to a 3.x version of the Python interpreter. On Mac it's similar I guess. On Windows you should use the programm py, which is installed together with newer python versions. I think it's delivered since Python 3.5.

This helper command finds the right interpreter for you. You just specify which version you want to have.

from subprocess import Popen, PIPE


def run_script(script_path, python_version=3, encoding='utf8'):
    return Popen(['py', f'-{python_version}', script_path], stdout=PIPE, stderr=PIPE, encoding=encoding)

To use Python 27, version have to be 27. If you want to have any available Python 2 version, you specify just 2 as version number.
The same with Python 3. If you want to have a Python 3 interpreter, just use 3 as version. Python 3.7.x is version 37. The dot is not used.

Another error is in the file "/sdc1/Apps/Modeling/FileEditing/untitled1m2.py", line 12

proc = subprocess.Popen(command, stdout=subprocess.PIPE, stdout=subprocess.PIPE)

2 times stdout. The second one should be stderr. Just a copy paste error.

Additionally I added encoding to subprocess. Then you get native strings instead of bytes.
With no specification of encoding, you get raw-bytes. This is interesting, if a program sends binary data to stdout.
You may have to change the encoding to another, if you use Python 2.

[stephenmolnar]

This code now works without any complaints:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Created on Mon May 13 09:50:54 2019

@author: comp
"""

import subprocess
with open('ligand') as infile:
ligand = infile.read().strip().split()
for nvar in ligand:
command = [
"./pythonsh", "./prepare_ligand4.py",
"-l", nvar + ".mol2",
"-o", nvar + ".pbdqt" ]
proc = subprocess.Popen(command, stdout=subprocess.PIPE)
out, err = proc.communicate()
if proc.returncode:
print ('Subprocess FAILED:', proc.command)

Many thanks for the help.