Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Upload takes too long.
#21
Yep. That is it. Select UTC i.e. Grenwich Mean. Of course, if this software does any plots/displays or such for you, they may be labeled in UTC instead of local time...
Reply
#22
Ok, set to UTC now.
Reply
#23
[Image: 20200403-173829.png]
Reply
#24
Here is my example of just reading the csv into values and parsing the date. I have added code that will abort if it tries to read more than one line from a csv. Note that your code reads each line of the csv and tries to upload each line to WOW without any pauses, because your requests call is inside the loop reading your csv file.
import csv
import datetime

with open('test.csv', 'r') as csv_file:
    csv_reader = csv.reader(csv_file)
    total_rows = 0
    for row in csv_reader:
        total_rows +=1
        assert total_rows == 1
        year = row[1]
        day_of_year = row[2]
        hhmm = row[3]
        hh = hhmm[:2]
        mm = hhmm[2:]
        winddir = row[12]
        windspeedmph = row[17]
        windgustmph = row[19]
        windgustdir = row[16]
        humidity = row[8]
        dewptf = row[7]
        tempf = row[5]
        rainin = row[27]
        dailyrainin = row[25]
        baromin = row[10]
        solarradiation = row[23]
        l = row[16]
softwaretype=1
year = int(year)
day_of_year = int(day_of_year) - 1
hour = int(hhmm[:2])
minute = int(hhmm[2:])
first_day_of_year = datetime.date(year, 1, 1)
dateutc = first_day_of_year + datetime.timedelta(days=day_of_year)
dateutc = datetime.datetime.combine(dateutc, datetime.time(hour, minute))
dateutc_string = f'&dateutc={dateutc:%Y-%m-%d+%H:%M:%S}'
dateutc_string = dateutc_string.replace(':', '%3a')
Finally, it does not make sense to have a python script running continuously for a scheduled task. If you are on windows you should set up a scheduled task that runs your python code every five minutes. On linux you would have to use crontab or one of its more modern variants. Personally, as someone who works regularly with ocean and met data, five minutes is overkill. If you change your AWS output frequency to 10 minutes and your upload frequency to ten minutes, you will never be rejected by WOW. You trade frequency for reliability. Also, if you are clever, you can catch when the WOW upload fails, wait five minutes and try the same upload string again. You get two chances over ten minutes to upload. If you choose 15 minutes, you could have three upload attempts.

Finally (I am full of hot air...), I would suggest using pywws instead of a home-baked solution.
Reply
#25
Hi,

Thanks alot for your input and effort.
I will have a look if i can get this working in my script.

Thanks.
Reply
#26
Almost forgot. My code assumes the time in your file is always four digits e.g. 0930, not 930. If this is not the case time will not be parsed properly for times between 0000 and 1200.
Reply
#27
Hi,

Thanks.
You mentioned to maybe use the windows task schedular option.
I was wondering the following regarding the above:

1. Is it easier to make a batch file to start the script?
2. When the task has run every 5 or so minutes, will the script window close or do i get multiple windows?
Reply
#28
You can can your python script directly from the scheduled task. Scheduled task also has some nice features like a timeout. You can tell the Task Scheduler to kill the job if it runs for longer than x seconds. I would set it to something like 4 minutes and 30 seconds to ensure it is dead before task instance starts. You also have an option for whether it runs if it missed running due to computer being off etc. I would turn that off. It might even have an option for only running a single instance of your task, which would be ideal. I am on linux now so can't check. You can also save an XML file describing the task, which you can reload into task manager if you have an accident, such as deleting your task.

Google a tutorial on Windows Scheduled Tasks. Basically make sure to include the full path to python, if you did not set on install. If python is in your windows PATH, your command would be python3 myscript.py if you have set the Task folder to the folder your script is in. You do not need a batch file.
Reply
#29
Ok thanks alot Smile
Reply
#30
Hi,

I have a question about the output CSV file.
When i open the CSV file with the data in Excel, it stopped updating the file.
Is it possible to have it run in Excel while updating and continue updating so the data is also updating in Excel?

Ah i found it....
But i see when i select 2 cells in Excel, the statusbar doesnt show average sum etc.
And when i double select/mouseclick those 2 cells the status bar is working.
Untill the data is refreshed, then it doesnt work again.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Run a timer when a functions starts to see how long the function takes to complete Pedroski55 2 1,986 Apr-19-2020, 06:28 AM
Last Post: Pedroski55

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020