os.system doesn't generate same result as shell?

[abc12346]

HI, I am a newbie of python and tried to use mixed python and shell code in my python script to achieve my goal. Therefore I use quite a lot of os.system() and/or subprocess.call(). I found that for some commands, the python generate different output from run shell directly. For example:

I have a source text file sample.txt which has lines ended with ":". I'd like to combine this kind of line with the next line. Since I know how to do this easily using vim, like "vim -e -c '%s/:\n/:/' -c 'wq' sample.txt", in my python code, I tried to os.system("vim -e -c '%s/:\n/:/' -c 'wq' sample.txt"), but the substitution did not happen. It looks like the "\n" is not recognized (it works when '%s/:/:ABC/'). Same behavior for subprocess.call("vim -e -c '%s/:\n/:/' -c 'wq' sample.txt", shell=True).

So my question: is os.system(shell_cmd) supposed to generate same result as running the shell_cmd directly? Any way to get around the above problem, yet still use vim instead of new python code?

My environment: python 2.7, bash under linux.

Thanks.
--Karl

[micseydel]

It's probably because a newline character, rather than the two \n characters, are being passed to vim. Instead of

os.system("vim -e -c '%s/:\n/:/' -c 'wq' sample.txt")

try

os.system(r"vim -e -c '%s/:\n/:/' -c 'wq' sample.txt")

This is called a "raw" string. Essentially, instead of Python helpfully turning the \n into a newline for you, it leaves those two characters as-is.

You'll probably need to deal with this quite a bit in the workflow you currently have. An alternative that you might like is something like

with open("sample.txt") as f:
  contents = f.read()

with open("sample.txt", "w") as f:
  f.write(contents.replace(":\n", ""))

(Note that vim may very well be smart enough to not load the entire file into memory. My Python here is naive.)