looking for a module

[Skaperen]

i would like to find, if one exists, a module that can make it easy to write small and simple scripts to make simple scripted edits to text files. i want to be able to do edits like "replace all instances of 'foo' to 'bar' between lines 30 and 70" or "replace all instances of the string between 'foo' and 'bar' with 'woot' on the lines between where 'fred' and 'flintstone' are found".

[snippsat]

I don't now of any module that dos this specific task.
It should't be to hard to write something that dos this with a few lines.
Something like this.

replace all instances of 'foo' to 'bar' between lines 30 and 70

from itertools import islice

with open('input.txt') as f:
    for line in islice(f, 30, 70):
        print(line.strip().replace('foo', 'bar'))

replace all instances of the string between 'foo' and 'bar' with 'woot'

flag = 1
with open('input.txt') as f:
    for line in f:
        if 'foo' in line:
            flag = 0
        if 'bar' in line:
            flag = 1
        if not flag and not 'foo' in line:
           line = 'Woot'
           print(line.strip())
        else:
            print(line.strip())

Output:
1 line 1
2 line 2
3 line foo here
Woot
Woot
Woot
7 line bar here
8 line 8
9 line 9

[ljmetzger]

UNIX like sed can do what you want from a Windows (or Linux) script (Tested on Windows 10 only):

sed -i "30,70 s/foo/bar/g" abc.txt (Linux may need single quotes)

sed can be run from a Python Script as follows:

# sed.exe example (sed = UNIX stream editor)

# Download (Windows version) : http://unxutils.sourceforge.net/  (extract sed.exe from .zip file)
# Intro and Tutorial: http://www.grymoire.com/Unix/Sed.html
# Using sed from Python: https://askubuntu.com/questions/747450/how-do-i-call-a-sed-command-in-a-python-script

# -i = edit in place (i.e. replace original file)
#  s  = substitute
#  g  = global (i.e. replace all occurrences on each line)

# Windows cmd.exe call string:
# sed.exe -i "30,70 s/foo/bar/g" abc.txt 

import subprocess
myresult = subprocess.call(["sed", "-i",  r"30,70 s/foo/bar/g", "abc.txt"])

Anecdotal evidence suggests that native Python code is superior to sed such as snippsat's example code or similar code using regular expressions.

I have not used sed in many years, but if several different similar edits are to be done on a group of files, my personal preference would be to use sed and a Windows Command file (or Linux script) obtaining the commands from a sed input file. I think debugging the changes would be much quicker in sed than using Python.

NOTE: Linux probably does not need a download. sed probably comes with Linux.

Lewis

[Gribouillis]

You can also try a python tool similar to sed: pyp (this is a package in ubuntu). Try pyp --manual or pyp -h or man pyp.

There is a introductory video here

[Skaperen]

sed does come with the Linux distros i have used, like Centos, Fedora, Slackware, and Ubuntu. but sed has its limitations. sed (now days) has an option -i that applies the edit to a file in place. i wrote a wrapper (named "sedi") around that which applies the edit to multiple files.

but what i want for python is something that handles files better, writing them back in place safely (write to a temp file, hard link the old file, move the new file), keep original owner and group (when run as root), keep original permissions, and edit multiple files without the edit code needing to know. the edit code would not even need to do an open.

i did this once in C and made it work by having the edit code call a start function, getting a reference as the return value, perform all edits on "one" file, and call a final function. the "magic" was the start function would return multiple times, once for each file, so it was not necessary to code the editing in a function.

i've also done that "magic" as a function to listen for network connections. it would return multiple times, once for each incoming connection.

for the editor, i'm also thinking of a bunch of functions for common editing steps like moving code around. find a location by some means and hold on to that reference. find a block of text by some means (two searches), and move that block to the saved location.

---

just thinking it over ... such "magic" might be doable as a generator.