Python Forum - Saving the times a script is run to a file or ...

Hi,

I have a lot of scripts from which I'd like to know the amount of times it was run.

I coded a working version a long time ago plus I was even more a beginner than I'm currently. So I'll show you what I have, then perhaps you have a better understanding.

Scope:
- several .py-files
- on network location (local)
- multiple users run those files.
- all of these files run my "log" function at the end of the file.
- log function saves to a network .txt file (simple dump).

Function:

def log():   
    import __main__ 
    project_nummer = projectnumber()
    date = datetime.now().strftime("%d %B %Y")
    dyn_naam = os.path.basename(__main__.__file__)[:-10]
    
    log_path = "T:\\Folder1\\Folder2\\myscripts.log\\"
         
    bestand = log_path + "mylog.txt"
    
    input = dyn_naam + "\t" + project_nummer + "\t" + date + "\n"
    exists = os.path.isfile(bestand)
    
    if exists:
        with open(bestand, "a+") as f:	
            f.writelines(input)	
        return "Log: {} {}".format(log_path, input)
    else:
        print "No log file exists"

Log file:
pdf 3473 19 February 2020
pdf 3473 19 February 2020
stempel 3496 19 February 2020
status 3496 19 February 2020

My questions:
- which is the better format? I now would say json due the library style
- what if two users run (different) scripts at the same time. Could the file be 'open' thus creating an error? If so: can I catch that?
- is this saving to a network location a good approach?

(Oct-18-2022, 06:25 AM)3Pinter Wrote: [ -> ]My questions:
- which is the better format? I now would say json due the library style
- what if two users run (different) scripts at the same time. Could the file be 'open' thus creating an error? If so: can I catch that?
- is this saving to a network location a good approach?

"Which is the better format?" I would not choose json because a json file is supposed to contain one object and is not fit for appending. There are ways to fiddle about with the json principle but I would not do so. I would prefer a csv file.
Concurrency and file locking: that is an interesting question. A file object has a "closed" attribute which holds the value True or False, but I suspect this covers only the situation in one program. Not concurrent programs. You could do a test with one program opening a file for let's say one minute, and then a second program trying to open the same file. Does it produce an exception? Then you could use an try ... except construction to wait until the file is not locked anymore. If you are really concerned about this you should consider to use a database. A database will have mechanisms to handle concurrent updates. Even sqlite3 (included with Python) has these mechanisms.
I would certainly prefer a network location.

json or csv comes down to how you want the data organized. Is it just a list of "script, datetime", then csv is the best choice. If it is "script:[datetimes]" then json is better. CSV is always nice for viewing.

If you are worried about multiple computers updating the value at the same time, I would make it a database.

well,

It's never 'updating' a line.
it's just adding and adding and adding.--> "script X, projectnumber, date"

I could rewrite it to csv, no problem.

Unsure how to test the 'same time' two scripts wanting to write something in the text (or csv) file.

(Oct-18-2022, 04:26 PM)3Pinter Wrote: [ -> ]Unsure how to test the 'same time' two scripts wanting to write something in the text (or csv) file.

First: if you are planning to have the csv file on a share, then test it on that share.
Create two scripts:
First script:

import time

myfile = r"\\myserver\myshare\myfolder\testbestand.txt"
testbestand = open(myfile, "a")
print("This is the first script", file=testbestand)
time.sleep(60)
print("First script closes file", file=testbestand)
testbestand.close()

Second script:

myfile = r"\\myserver\myshare\myfolder\testbestand.txt"
testbestand = open(myfile, "a")
print("This is the second script", file=testbestand)
testbestand.close()

Start the first script. It will run one minute. Within this minute start the second script. Does it give an error? What are the contents after both scripts have run? And what happens if you start the two scripts from different computers?

I can't test this myself because I don't have Windows.

Ahhh that's clever! Certainly something I can test.

I did test this one:
- opening the txt file using notepad
- run script.
result: no errors, reopening txt file showed the correct file (with the added line).

Thanks guys!

(Oct-19-2022, 07:36 AM)ibreeden Wrote: [ -> ]
(Oct-18-2022, 04:26 PM)3Pinter Wrote: [ -> ]Unsure how to test the 'same time' two scripts wanting to write something in the text (or csv) file.
First: if you are planning to have the csv file on a share, then test it on that share.
Create two scripts:
First script:
import time

myfile = r"\\myserver\myshare\myfolder\testbestand.txt"
testbestand = open(myfile, "a")
print("This is the first script", file=testbestand)
time.sleep(60)
print("First script closes file", file=testbestand)
testbestand.close()
Second script:
myfile = r"\\myserver\myshare\myfolder\testbestand.txt"
testbestand = open(myfile, "a")
print("This is the second script", file=testbestand)
testbestand.close()
Start the first script. It will run one minute. Within this minute start the second script. Does it give an error? What are the contents after both scripts have run? And what happens if you start the two scripts from different computers?

I can't test this myself because I don't have Windows.

(Oct-19-2022, 03:28 PM)3Pinter Wrote: [ -> ]I did test this one:
- opening the txt file using notepad
- run script.

Be careful. As far as I know Notepad.exe does not keep the file open. It opens the file, reads the content in memory and closes the file. When saving also the file is kept open short.

(Oct-19-2022, 04:14 PM)ibreeden Wrote: [ -> ]
(Oct-19-2022, 03:28 PM)3Pinter Wrote: [ -> ]I did test this one:
- opening the txt file using notepad
- run script.
Be careful. As far as I know Notepad.exe does not keep the file open. It opens the file, reads the content in memory and closes the file. When saving also the file is kept open short.

Notepad++ does that to my understanding, didn't realise notepad does too. Good advise. Will do some more testing upcoming weekend (rl. is bugging me atm sigh)