Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
temporary file names
#1
Python does have functions to create temporary files that can be used for temporary storage. but what i am looking for it a function to create a file that will have a temporary name and is destined to be renamed or relinked to become a file named with the given name. the caller will do the rename if it is appropriate or will remove the file. a use case would be when the caller is making a change to the contents of the file and must have no point in time when the file could be lost.

my intent is to encapsulate this as a class so that details of relinking, and finding an appropriate name that can be renamed are not dealt with by the caller. instead the caller would finish up by calling a .finish() method with a boolean to indicate if the new file is to be kept.

but i don't want to code this if it is in Python, already.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
It's a great idea. I suggest a context manager and a 'commit' keyword argument to the .close() method in order to have code such as
with skap_open("critical.cfg", "w") as ofile:
    ofile.write("frac = {}\n".format(x / y))
    ofile.write("more data\n")
    ofile.close(commit=True)
If we reach the ofile.close(commit=True) statement, critical.cfg is written. If we exit through an exception or some other way, the temporary file is destroyed but critical.cfg is left untouched. The commit keyword defaults to False, so that exiting the context defaults to not overwriting 'critical.cfg'.
Reply
#3
the idea i have is that the file is written, but to the alternate name. then when a .close() is done, the alternate file is closed, then it is renamed to the original name, replacing it. this is created by specifying a name to write to. if that name does not exist, it is opened in a normal way. but if it does exist, this code determines an alternate file name that can be renamed to the original an stay in the same filesystem. it's how editors typically save files.

there would be an option to abandon the file instead of close it. another method would get a copy of the alternate name.

i need better terminology for this.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#4
Here is my first proposal for this functionality. I'm using a .commit() method to actually overwrite the file.
# skapopen.py
# safely write to file by hiding a temporary file
from contextlib import contextmanager
from functools import partial
import os
from pathlib import Path
import shutil
import tempfile
__version__ = '0.0.2'

class SkapOpenError(RuntimeError): pass

def valid_path(filename):
    p = Path(filename)
    if not p.is_absolute():
        p = Path.cwd()/p
    # raise error if parent dir doesn't exist (FileNotFoundError(OSError))
    parent = p.parent.resolve()
    if not parent.is_dir():
        raise SkapOpenError('Not a directory', str(parent))
    return parent/p.name

def close(fh):
    fh.close()

def noop():
    pass

def commit(targetfile, tfh):
    tfh.close()
    shutil.move(tfh.name, targetfile)
    tfh.commit = noop

@contextmanager
def skapopen(filename, mode, *args, **kwargs):
    if not 'w' in mode:
        raise SkapOpenError('Writing mode expected, got', mode)
    p = valid_path(filename)
    if p.exists():
        if not (p.is_file() and not p.is_symlink()):
            raise SkapOpenError('Not a regular file', str(p))
        kwargs['delete'] = False
        with tempfile.NamedTemporaryFile(mode=mode, *args, **kwargs) as tfh:
            try:
                tfh.commit = partial(commit, str(p), tfh)
                yield tfh
                tfh.close()
            finally:
                try:
                    os.remove(tfh.name)
                except OSError:
                    pass
    else:
        with open(str(p), mode=mode, *args, **kwargs) as fh:
            fh.commit = partial(close, fh)
            yield fh


if __name__ == '__main__':
    # TEST CODE
    testfile = 'tmp_foo_foo_foo.txt'
    def check_content(value):
        with open(testfile) as ifh:
            assert ifh.read() == value
    def remove_testfile():
        try:
            os.remove(testfile)
        except OSError:
            pass
    remove_testfile()
    # write to a non existing file
    with skapopen(testfile, 'w') as ofh:
        ofh.write('hello world\n')
    check_content('hello world\n')
    # write to an existing file and don't commit
    with skapopen(testfile, 'w') as ofh:
        ofh.write('lorem ipsum\n')
    check_content('hello world\n')
    # write to a existing file and commit
    with skapopen(testfile, 'w') as ofh:
        ofh.write('Ille homo\n')
        ofh.commit()
    check_content('Ille homo\n')
    remove_testfile()
    # write to a non existing file and commit
    with skapopen(testfile, 'w') as ofh:
        ofh.write('It is awesome\n')
        ofh.commit()
    check_content('It is awesome\n')
    remove_testfile()
There is still some work to do, especially concerning the args and kwargs of skapopen(). Also one should perhaps work in the direction of your 'same filesystem' request. Perhaps use the 'dir' argument in NamedTemporaryFile?
Reply
#5
this grew out of a project to just figure out an alternate name to use that was in the same filesystem. it would not necessarily be in the same directory. my logic would check the parents (and up) of the named file plus the current working directory (and its parents) for subdirectories named "tmp" or the parents themselves in the 2nd pass, that were in the same filesystem.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#6
Skaperen Wrote:my logic would check the parents (and up) of the named file plus the current working directory (and its parents) for subdirectories named "tmp" or the parents themselves in the 2nd pass, that were in the same filesystem.
You could generate candidate 'tmp' directories on the same device as Path p by using this function (which you can call after using valid_path() in the above code)
def tmp_candidates(p):
    dev = p.stat().st_dev
    for x in [p.parent, Path.cwd()]:
        while True:
            tmp = x/'tmp'
            if tmp.is_dir() and tmp.stat().st_dev == dev:
                yield tmp
            x, y = x.parent, x
            if x == y:
                break
You can then select the 'tmp' directory with
tmp = next(tmp_candidates(p), None)
if tmp is None:
    # no tmp directory found on the same device. Perhaps create such a directory or use a
    # temporary directory on another device or the system default
    do_something()
Reply
#7
if i can create a "tmp" directory then i can create the alternate file as its alternate name. if something exists as that alternate name, then i need to generate another, better, alternate name. i typically put a timestamp in such alternate names. if the logic to regenerate is in place, than the first one has a lower resolution like seconds and the regenerated one has higher resolution like microseconds+pid.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  rename same file names in different directories elnk 0 680 Nov-04-2022, 05:23 PM
Last Post: elnk
  Saving Excel workbook file with dataframe names Biplab1985 0 1,996 Jun-07-2020, 12:25 PM
Last Post: Biplab1985
  Details of attachment files in a msg file such as file names save into a python list klllmmm 2 5,625 Nov-12-2019, 05:59 AM
Last Post: klllmmm
  splitstring file names a by hyphen steve22020 3 3,236 Oct-30-2019, 05:39 PM
Last Post: steve22020
  Is there a more effecient way to do this ? File Names sumncguy 2 2,044 Jul-11-2019, 12:47 PM
Last Post: DeaD_EyE
  How to combine file names into a list from multiple directories? python_newbie09 3 5,137 Jul-09-2019, 07:38 PM
Last Post: python_newbie09
  write each line of a text file into separate text files and save with different names Shameendra 3 2,745 Feb-20-2019, 09:07 AM
Last Post: buran
  Reading file names CertifiedPengu 1 2,252 Jan-07-2019, 06:54 PM
Last Post: Gribouillis
  how to increment all file names in a folder SoulsKeeper 5 5,947 Sep-10-2018, 07:59 PM
Last Post: Skaperen
  dealing with spaces in file names AceScottie 5 74,649 Jun-02-2018, 01:06 PM
Last Post: AceScottie

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020