"The problem is that the enormous data set text files I copy/paste from have dates and values but no commas."
Why do you copy data (manually)? Where do you paste it?
There is no such datastructure in Python:
2009 09 23 11 45 00 1 9999.000
2009 09 23 12 00 00 1 9999.000
2009 09 23 12 15 00 1 4593.017
2009 09 23 12 30 00 1 4593.005
2009 09 23 12 45 00 1 4592.993
What is it? String? Multiline string? Text file?
If data is in file then one should iterate over it row by row.
Snippet of code I provided is working example how to solve this type of problems. If you don't know how to use it then you should say so. Nobody is 'playing' here.
How big is your file? Or there are many of them? Appr how many rows? Is my calculation correct:
EDIT:
Let's assume that I have a file which has rows as you provided in raw data and this is stored in file named buoys.txt, then I could do:
Why do you copy data (manually)? Where do you paste it?
There is no such datastructure in Python:
2009 09 23 11 45 00 1 9999.000
2009 09 23 12 00 00 1 9999.000
2009 09 23 12 15 00 1 4593.017
2009 09 23 12 30 00 1 4593.005
2009 09 23 12 45 00 1 4592.993
What is it? String? Multiline string? Text file?
If data is in file then one should iterate over it row by row.
Snippet of code I provided is working example how to solve this type of problems. If you don't know how to use it then you should say so. Nobody is 'playing' here.
How big is your file? Or there are many of them? Appr how many rows? Is my calculation correct:
>>> 4000 * 96 * 365 * 20 # number of buoys * water height measurements per day * days in year * years 2803200000Do you need only two numbers out of all these rows?
EDIT:
Let's assume that I have a file which has rows as you provided in raw data and this is stored in file named buoys.txt, then I could do:
with open('buoys.txt', 'r') as f: rows = (row.strip() for row in f) nums = (float(row.split()[-1]) for row in rows if float(row.split()[-1]) != 9999.0) max_value = max(nums)This should be gentle on memory as it's generator piping i.e. rows are consumed one at the time.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy
Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.