Python does have functions to create temporary files that can be used for temporary storage. but what i am looking for it a function to create a file that will have a temporary name and is destined to be renamed or relinked to become a file named with the given name. the caller will do the rename if it is appropriate or will remove the file. a use case would be when the caller is making a change to the contents of the file and must have no point in time when the file could be lost.
my intent is to encapsulate this as a class so that details of relinking, and finding an appropriate name that can be renamed are not dealt with by the caller. instead the caller would finish up by calling a .finish() method with a boolean to indicate if the new file is to be kept.
but i don't want to code this if it is in Python, already.
It's a great idea. I suggest a context manager and a 'commit' keyword argument to the
.close() method in order to have code such as
with skap_open("critical.cfg", "w") as ofile:
ofile.write("frac = {}\n".format(x / y))
ofile.write("more data\n")
ofile.close(commit=True)If we reach the
ofile.close(commit=True) statement, critical.cfg is written. If we exit through an exception or some other way, the temporary file is destroyed but critical.cfg is left untouched. The commit keyword defaults to False, so that exiting the context defaults to not overwriting 'critical.cfg'.
the idea i have is that the file is written, but to the alternate name. then when a .close() is done, the alternate file is closed, then it is renamed to the original name, replacing it. this is created by specifying a name to write to. if that name does not exist, it is opened in a normal way. but if it does exist, this code determines an alternate file name that can be renamed to the original an stay in the same filesystem. it's how editors typically save files.
there would be an option to abandon the file instead of close it. another method would get a copy of the alternate name.
i need better terminology for this.
Here is my first proposal for this functionality. I'm using a .commit() method to actually overwrite the file.
# skapopen.py
# safely write to file by hiding a temporary file
from contextlib import contextmanager
from functools import partial
import os
from pathlib import Path
import shutil
import tempfile
__version__ = '0.0.2'
class SkapOpenError(RuntimeError): pass
def valid_path(filename):
p = Path(filename)
if not p.is_absolute():
p = Path.cwd()/p
# raise error if parent dir doesn't exist (FileNotFoundError(OSError))
parent = p.parent.resolve()
if not parent.is_dir():
raise SkapOpenError('Not a directory', str(parent))
return parent/p.name
def close(fh):
fh.close()
def noop():
pass
def commit(targetfile, tfh):
tfh.close()
shutil.move(tfh.name, targetfile)
tfh.commit = noop
@contextmanager
def skapopen(filename, mode, *args, **kwargs):
if not 'w' in mode:
raise SkapOpenError('Writing mode expected, got', mode)
p = valid_path(filename)
if p.exists():
if not (p.is_file() and not p.is_symlink()):
raise SkapOpenError('Not a regular file', str(p))
kwargs['delete'] = False
with tempfile.NamedTemporaryFile(mode=mode, *args, **kwargs) as tfh:
try:
tfh.commit = partial(commit, str(p), tfh)
yield tfh
tfh.close()
finally:
try:
os.remove(tfh.name)
except OSError:
pass
else:
with open(str(p), mode=mode, *args, **kwargs) as fh:
fh.commit = partial(close, fh)
yield fh
if __name__ == '__main__':
# TEST CODE
testfile = 'tmp_foo_foo_foo.txt'
def check_content(value):
with open(testfile) as ifh:
assert ifh.read() == value
def remove_testfile():
try:
os.remove(testfile)
except OSError:
pass
remove_testfile()
# write to a non existing file
with skapopen(testfile, 'w') as ofh:
ofh.write('hello world\n')
check_content('hello world\n')
# write to an existing file and don't commit
with skapopen(testfile, 'w') as ofh:
ofh.write('lorem ipsum\n')
check_content('hello world\n')
# write to a existing file and commit
with skapopen(testfile, 'w') as ofh:
ofh.write('Ille homo\n')
ofh.commit()
check_content('Ille homo\n')
remove_testfile()
# write to a non existing file and commit
with skapopen(testfile, 'w') as ofh:
ofh.write('It is awesome\n')
ofh.commit()
check_content('It is awesome\n')
remove_testfile()There is still some work to do, especially concerning the args and kwargs of
skapopen(). Also one should perhaps work in the direction of your 'same filesystem' request. Perhaps use the 'dir' argument in NamedTemporaryFile?
this grew out of a project to just figure out an alternate name to use that was in the same filesystem. it would not necessarily be in the same directory. my logic would check the parents (and up) of the named file plus the current working directory (and its parents) for subdirectories named "tmp" or the parents themselves in the 2nd pass, that were in the same filesystem.
Skaperen Wrote:my logic would check the parents (and up) of the named file plus the current working directory (and its parents) for subdirectories named "tmp" or the parents themselves in the 2nd pass, that were in the same filesystem.
You could generate candidate 'tmp' directories on the same device as Path p by using this function (which you can call after using
valid_path() in the above code)
def tmp_candidates(p):
dev = p.stat().st_dev
for x in [p.parent, Path.cwd()]:
while True:
tmp = x/'tmp'
if tmp.is_dir() and tmp.stat().st_dev == dev:
yield tmp
x, y = x.parent, x
if x == y:
breakYou can then select the 'tmp' directory with
tmp = next(tmp_candidates(p), None)
if tmp is None:
# no tmp directory found on the same device. Perhaps create such a directory or use a
# temporary directory on another device or the system default
do_something()
if i can create a "tmp" directory then i can create the alternate file as its alternate name. if something exists as that alternate name, then i need to generate another, better, alternate name. i typically put a timestamp in such alternate names. if the logic to regenerate is in place, than the first one has a lower resolution like seconds and the regenerated one has higher resolution like microseconds+pid.