Python Forum
sub command - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: General (https://python-forum.io/forum-1.html)
+--- Forum: Code sharing (https://python-forum.io/forum-5.html)
+--- Thread: sub command (/thread-24859.html)



sub command - Skaperen - Mar-07-2020

sub.py is my "sub" command. it substitutes one string (arg 2) for another (arg 1) in one or more files or in th stdin/stdout stream for file name "-" (or if no names given).
#!/usr/bin/env python3
from os import rename,remove
from os.path import exists
from subprocess import run
from sys import argv,stderr,stdin,stdout
from time import time as secs
dne = 0
abort = False
if len(argv)<3:
    exit('string substition needs old string in arg 1 and new string in arg2 followed by an optional list of file names.')
exe = argv.pop(0)
old = argv.pop(0)
new = argv.pop(0)
fns = argv if argv else ['-']
for fn in fns:
    if fns.count(fn)>1:
        exit('duplicate file name: '+repr(fn))
    if fn!='-':
        if not exists(fn):
            print('file does not exist: '+repr(fn),file=stderr)
            dne += 1
if dne:
    exit('aborting due to '+str(dne)+' missing file'+'s'[dne==1:])
for fn in fns:
    if fn=='-':
        tn = None
        fi,fo = stdin,stdout
    else:
        tn = fn+'+'+str(int(secs()*3906250))
        fi = open(fn,'r')
        fo = open(tn,'w')
    changed = 0
    for ol in fi:
        nl = ol.replace(old,new)
        if nl!=ol:
            changed += 1
        fo.write(nl)
    fi.close()
    fo.close()
    if changed and fn!='-':
        run(['chmod','--quiet','--reference='+fn,tn])
        run(['chown','--quiet','--reference='+fn,tn])
        rename(fn,fn+'~')
        rename(tn,fn)
    else:
        if tn:
            remove(tn)



RE: sub command - Gribouillis - Mar-07-2020

I would attempt to replace lines 24-end with
for fn in fns:
    with fileinput.input((fn,), inplace=True, backup='~') as input:
        for line in input:
            print(line.replace(old, new), end='')



RE: sub command - Skaperen - Mar-08-2020

i don't follow what those 4 lines are doing, especially the print() call. the 1st line is clear. i'll have to guess regarding the 2nd line. i presume the 3rd iterates each line. how does print() get its output to the file? what if the file had nothing to be changed?


RE: sub command - Gribouillis - Mar-08-2020

Sorry, I forgot the fileinput.input(...) in the first version. You need to import the fileinput module first.

It's all explained in the fileinput's module documentation

documentation Wrote:Optional in-place filtering: if the keyword argument inplace=True is passed to fileinput.input() or to the FileInput constructor, the file is moved to a backup file and standard output is directed to the input file (if a file of the same name as the backup file already exists, it will be replaced silently). This makes it possible to write a filter that rewrites its input file in place. If the backup parameter is given (typically as backup='.<some extension>'), it specifies the extension for the backup file, and the backup file remains around; by default, the extension is '.bak' and it is deleted when the output file is closed. In-place filtering is disabled when standard input is read.

Within the 'with' block, writing to standard output will actually write to the target file. It gives a shorter code because we don't need to open the output file explicitly and the creation of the backup file is also handled by fileinput.input() If there is no change, I think the file is simply copied.


RE: sub command - buran - Mar-08-2020

Also, better not use input as file handler :)


RE: sub command - Gribouillis - Mar-08-2020

I have not tested this but one can probably even remove the for loop and write directly this
with fileinput.input(fns, inplace=True, backup='~') as ifh:
    for line in ifh:
        print(line.replace(old, new), end='')
Suggestions and wish list:
  • Write a series of unittests for this program.
  • Handle arguments with specialized module such as argparse or click or plumbum.cli.

(I followed buran's advice although the official documentation doesn't!)


RE: sub command - ndc85430 - Mar-08-2020

The program is already reasonably complex, so testing it well is going to be difficult. Separation of concerns is important - split it up into classes and functions that can be understood and tested independently (this also helps with maintenance of the program, of course). You still need to test that those components have been integrated properly - that the application as a whole works, so you'd have some high level acceptance tests to do that (though fewer in number than the unit tests, in accordance with the test pyramid).


RE: sub command - Skaperen - Mar-08-2020

because rewriting the file updates the file's timestamp, my program explicitly avoids the file rename/move and, instead, deletes the temporary file. does fileinput check for unchanged content?