Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Restarting a script
#1
Hi everyone,

I'm quite new to coding and I currently have a fairly simple python script that I run that queries an API based on an input file I provide it and then sends me back a new file (GIS image). If the script is interrupted, it creates a log file, so that when I start it again, it knows where to begin (rather than reprocessing all 1000 input lines) I run this thousands of times for the project I'm working on. The issue is that my internet connection isn't great, so the script often dies and then I need to manually restart it, which is especially an issue when I'm sleeping as I can't manually restart it. I'm looking for a way to restart the script (including specifying the input file). Basically to run the script it's just "py .\script.py .\filename.csv 1", which I would have already used in command line to start the script.

I've looked into using a "while" loop but that doesn't quite work (at least how I've done it), since I only want the script to run again after it crashes (which it needs to do to produce the log it will use when it starts up again). Any ideas or suggestions greatly appreciated!

Here is the script below for reference:
import requests
import csv
import sys
import os
import time
import json

server="https://website.com/"

if len(sys.argv) == 1:
	print("ERROR: Need a .csv file\neg. python coverage.py mydata.csv #group")
	quit()

if not os.path.exists("calculations"+"_"+sys.argv[2]):
		os.makedirs("calculations"+"_"+sys.argv[2])

# Open CSV file
csvfile = csv.DictReader(open(sys.argv[1]))


log = open("log.csv",'a', newline='')
wrlog = csv.writer(log, dialect='excel')

errors = open("errors.csv",'a', newline='')
wrerror = csv.writer(errors, dialect='excel')

with open("log.csv", newline='') as f:
    #reader = csv.reader(f)
    done = list(f)
done_clean=[]
for place in done:
    done_clean.append(site.rstrip('\r\n'))


f = open(sys.argv[1])
numlines = len(f.readlines())
sites_error=[]
number_sites_done=0

n=0
for row in csvfile:
    if row['nam'] not in sites_done_clean:
        #time.sleep(6) # Pause script. Important otherwise server will ban you.
        start_time = time.time() # Stopwatch start
        r = requests.post(server+"/API/area", data=row)
        j=json.loads(r.text)
        if 'error' in j:
            print('Error : '+str(row['nam']))
            print(r.text)
            wrerror.writerow([row['nam'],r.text])
        else:
            if 'tiff' in j:
            	#print (j['tiff'])
                r = requests.get(j['tiff'])
                if len(r.content)>2500000:
                    fn="calculations_"+row['group']+os.sep+str(row['nam'])+".tiff"
                    filename = open(fn,"wb")
                    filename.write(r.content)
                    filename.close()
                    wrprocessed.writerow([row['nam']])
                    print("  "+str(n+1)+"/"+str(numlines-number_done)+" Place --> "+str(row['nam']))
                else:
                    print('File size error: '+str(row['nam'])+"  File Size: "+str(len(r.content)))
                    wrerror.writerow([row['nam'],"File size error. Size: "+str(len(r.content))])
        elapsed = round(time.time() - start_time,1) # Stopwatch
        n=n+1
    else:
        number_done+=1
Gribouillis write Nov-11-2020, 02:33 PM:
Please post all code, output and errors (it it's entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button.

I fixed for you this time. Please use code tags on future posts.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How do I properly implement restarting a multithreaded python application? MrFentazis 1 588 Jul-17-2023, 09:10 PM
Last Post: JamesSmith
  Skipping line in text without Restarting Loop IdMineThat 4 1,433 Apr-05-2022, 04:23 AM
Last Post: deanhystad
  sqlite3 database does not save data across restarting the program SheeppOSU 1 3,405 Jul-24-2020, 05:53 AM
Last Post: SheeppOSU
  Restarting code from scratch palladium 1 1,972 Feb-29-2020, 04:46 PM
Last Post: ibreeden
  py 2.7 restarting the program Jonathan_levy 1 2,279 Jul-06-2018, 07:44 AM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020