[SOLVED]passing a lot of information to subprocess.call() - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: [SOLVED]passing a lot of information to subprocess.call() (/thread-7826.html) Pages:
1
2
|
[SOLVED]passing a lot of information to subprocess.call() - Barrowman - Jan-26-2018 Hi, I have a long list of names and phone numbers in the following format: Quote:0100 12345I call this file first.txt What I am doing is create a new file which lists each name and records at the side of it how many times their name appears in the original list. What I have been doing is using a simple python script to generate a file I call second.txt. This just contains the names. From that I run the following: cat second.txt|sort|uniq>third.txtThis generates another file which has each name in alphabetical order once. Another python script then reads through third.txt and for each line reads through second.txt and to count how many times that name appears. After each search it writes to a new file this number with the name beside it. So what I wanted to do is to make one python script to do it all from start to finish rather than running the 3 separate commands. I tried os.system and then subprocess to incorporate the sort and uniq commands line but I cannot seem to get it to work. What ever I have tried gives syntax errors. For example cmd = "'cat', /home/norman/khconf/second.txt, '|sort', '|uniq', '>', /home/norman/khconf/third.txt" subprocess.call([cmd]) so then I tried:cmd = "'/bin/cat', /home/norman/khconf/second.txt, '|/usr/bin/sort', '|/usr/bin/uniq', '>', /home/norman/khconf/third.txt" subprocess.call([cmd])gave Can someone point me in the right direction please?
RE: passing a lot of information to subprocess.call() - Barrowman - Jan-26-2018 This is all running under python 3.5.2 Well after a lot more googling I have found a different approach instead of subprocess.call cat = subprocess.Popen(['cat', '/home/norman/khconf/second.txt'], stdout=subprocess.PIPE, ) sort = subprocess.Popen(['sort'], stdin=cat.stdout, stdout=subprocess.PIPE, ) uniq = subprocess.Popen(['uniq'], stdin=sort.stdout, stdout=subprocess.PIPE, ) end_of_pipe = uniq.stdout lastfile = docs + "/third.txt" lastone = open(lastfile, 'w') lastone.write("Hello World") for line in end_of_pipe: lastone.write(line)I get no errors but third.txt only contains the "Hello World" I modified the code from https://pymotw.com/2/subprocess/ Can anyone suggest how I can fix this please? RE: passing a lot of information to subprocess.call() - anapbevi - Jan-26-2018 What if you delete the last 3 lines? Won`t work? I m just guessing, cause I don t have the files to make a simulation...(First day at coding...) RE: passing a lot of information to subprocess.call() - wavic - Jan-26-2018 command = "cat /home/norman/khconf/second.txt | sort | uniq > /home/norman/khconf/third.txt".split() subprocess.call(command)When you open a file with 'w' flag for writing it overwrites the previous content. Use 'a' to append. https://docs.python.org/3/library/functions.html#open RE: passing a lot of information to subprocess.call() - Barrowman - Jan-26-2018 (Jan-26-2018, 09:44 PM)wavic Wrote:the syntax iscommand = "cat /home/norman/khconf/second.txt | sort | uniq > /home/norman/khconf/third.txt".split() subprocess.call(command) subprocess.call([command])When I run the command from the terminal it works fine without the .split Quote:When you open a file with 'w' flag for writing it overwrite the previous content. Use 'a' to append. Yes, but it is a file I am creating and it only opens a new file or clears the old contents. From then on it will automatically append lines until the file is closed. This is what I want it to do. If you look at my first post I use a python script to read first.txt and create second.txt. First.txt has 46 lines in it and second.txt has 23 lines in it. This is exactly as it should be. I am trying to get this all working as the input file will continue to grow and could end up with hundreds of lines in it RE: passing a lot of information to subprocess.call() - wavic - Jan-27-2018 If the 'command' is already a list of arguments created by split() why you closing it in another []s? Print 'end_of_pipe' to see if it contains something. RE: passing a lot of information to subprocess.call() - Barrowman - Jan-27-2018 (Jan-27-2018, 07:29 AM)wavic Wrote: If the 'command' is already a list of arguments created by split() why you closing it in another []s? Sorry wavic when I do command = "cat /home/norman/khconf/second.txt | sort | uniq > /home/norman/khconf/third.txt".split() subprocess.call(command)I get
RE: passing a lot of information to subprocess.call() - Barrowman - Jan-27-2018 I have found out how to do it now. fout="/home/norman/khconf/third.txt" fo = open(fout, 'w') cat = subprocess.Popen(['cat', 'second.txt'], stdout=subprocess.PIPE, ) sort = subprocess.Popen(['sort'], stdin=cat.stdout, stdout=subprocess.PIPE, ) uniq = subprocess.Popen(['uniq'], stdin=sort.stdout, stdout=fo ) RE: [SOLVED]passing a lot of information to subprocess.call() - wavic - Jan-27-2018 (Jan-27-2018, 09:09 AM)Barrowman Wrote: Sorry wavic when I do This is because the call method doesn't use the shell to execute a command. You have to pass shell=True as a parameter to the method. But this is not safe.
RE: [SOLVED]passing a lot of information to subprocess.call() - buran - Jan-27-2018 well, now you can learn how to do it with python, without system calls and without need of 2 intermediate files... from collections import Counter with open('first.txt', 'r') as f1: names = [ln.strip() for i, ln in enumerate(f1) if i%2] with open('fourth.txt', 'w') as f4: for key, value in sorted(Counter(names).items()): f4.write('{} {}\n'.format(key, value)) |