Python Forum
Thread Rating:
  • 1 Vote(s) - 1 Average
  • 1
  • 2
  • 3
  • 4
  • 5
sub command
#1
sub.py is my "sub" command. it substitutes one string (arg 2) for another (arg 1) in one or more files or in th stdin/stdout stream for file name "-" (or if no names given).
#!/usr/bin/env python3
from os import rename,remove
from os.path import exists
from subprocess import run
from sys import argv,stderr,stdin,stdout
from time import time as secs
dne = 0
abort = False
if len(argv)<3:
    exit('string substition needs old string in arg 1 and new string in arg2 followed by an optional list of file names.')
exe = argv.pop(0)
old = argv.pop(0)
new = argv.pop(0)
fns = argv if argv else ['-']
for fn in fns:
    if fns.count(fn)>1:
        exit('duplicate file name: '+repr(fn))
    if fn!='-':
        if not exists(fn):
            print('file does not exist: '+repr(fn),file=stderr)
            dne += 1
if dne:
    exit('aborting due to '+str(dne)+' missing file'+'s'[dne==1:])
for fn in fns:
    if fn=='-':
        tn = None
        fi,fo = stdin,stdout
    else:
        tn = fn+'+'+str(int(secs()*3906250))
        fi = open(fn,'r')
        fo = open(tn,'w')
    changed = 0
    for ol in fi:
        nl = ol.replace(old,new)
        if nl!=ol:
            changed += 1
        fo.write(nl)
    fi.close()
    fo.close()
    if changed and fn!='-':
        run(['chmod','--quiet','--reference='+fn,tn])
        run(['chown','--quiet','--reference='+fn,tn])
        rename(fn,fn+'~')
        rename(tn,fn)
    else:
        if tn:
            remove(tn)
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
I would attempt to replace lines 24-end with
for fn in fns:
    with fileinput.input((fn,), inplace=True, backup='~') as input:
        for line in input:
            print(line.replace(old, new), end='')
Reply
#3
i don't follow what those 4 lines are doing, especially the print() call. the 1st line is clear. i'll have to guess regarding the 2nd line. i presume the 3rd iterates each line. how does print() get its output to the file? what if the file had nothing to be changed?
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#4
Sorry, I forgot the fileinput.input(...) in the first version. You need to import the fileinput module first.

It's all explained in the fileinput's module documentation

documentation Wrote:Optional in-place filtering: if the keyword argument inplace=True is passed to fileinput.input() or to the FileInput constructor, the file is moved to a backup file and standard output is directed to the input file (if a file of the same name as the backup file already exists, it will be replaced silently). This makes it possible to write a filter that rewrites its input file in place. If the backup parameter is given (typically as backup='.<some extension>'), it specifies the extension for the backup file, and the backup file remains around; by default, the extension is '.bak' and it is deleted when the output file is closed. In-place filtering is disabled when standard input is read.

Within the 'with' block, writing to standard output will actually write to the target file. It gives a shorter code because we don't need to open the output file explicitly and the creation of the backup file is also handled by fileinput.input() If there is no change, I think the file is simply copied.
Reply
#5
Also, better not use input as file handler :)
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#6
I have not tested this but one can probably even remove the for loop and write directly this
with fileinput.input(fns, inplace=True, backup='~') as ifh:
    for line in ifh:
        print(line.replace(old, new), end='')
Suggestions and wish list:
  • Write a series of unittests for this program.
  • Handle arguments with specialized module such as argparse or click or plumbum.cli.

(I followed buran's advice although the official documentation doesn't!)
Reply
#7
The program is already reasonably complex, so testing it well is going to be difficult. Separation of concerns is important - split it up into classes and functions that can be understood and tested independently (this also helps with maintenance of the program, of course). You still need to test that those components have been integrated properly - that the application as a whole works, so you'd have some high level acceptance tests to do that (though fewer in number than the unit tests, in accordance with the test pyramid).
Reply
#8
because rewriting the file updates the file's timestamp, my program explicitly avoids the file rename/move and, instead, deletes the temporary file. does fileinput check for unchanged content?
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020