How to get past NaN value - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: How to get past NaN value (/thread-1209.html) |
How to get past NaN value - iFunKtion - Dec-13-2016 Hi there, I am trying to do a simple test for a NaN value in python3 pandas, but I can't get anything to work. This is what I want to do, x is a float of value anything up to 1396253.0 if x == nan: continue else rest of program...I can't find any sytax for it, this itself gives a syntax error, I have seen also isnan() and can't get that working either. Is there a simple solution to this, as it has been cropping up a lot. A null value would have been much easier to deal with. RE: How to get past NaN value - micseydel - Dec-13-2016 In general, please show a real snippet of code you've tried that reproduces the problem. In this case, "rest of program..." is clearly a syntax error, so we can't tell what might be going on when you say that; if I put that aside, I'd expect a NameError, not a SyntaxError. Here's a full session where isnan works for me
RE: How to get past NaN value - iFunKtion - Dec-13-2016 Ok, here is the code, I have tried to use the isnan, but it just throws back a TypeError: #!/usr/bin/env python3 import glob import math import pandas as pd import numpy as np def get_avitime(vbo): try: df = pd.read_csv(vbo, delim_whitespace=True, header=90) row = next(df.iterrows()) t = df.tail(2).avitime.values[0] return t except: pass def human_time(seconds): secs = seconds/1000 mins, secs = divmod(secs, 60) hours, mins = divmod(mins, 60) return '%02d:%02d:%02d' % (hours, mins, secs) def main(): path = 'Z:\\VBox_Backup\\**\\*.vbo' events = {} customers = {} for vbo_path in glob.glob(path, recursive=True): path_list = vbo_path.split('\\') event = path_list[2].upper() customer = path_list[3].title() avitime = get_avitime(vbo_path) if not avitime: # this is where I tried if math.isnan(float(avitime)): continue else: if event not in events: events[event] = {customer:avitime} print(event) elif customer not in events[event]: events[event][last_customer] = human_time(events[event][last_customer]) print(last_customer, events[event][last_customer]) # print(events[event][last_customer]) events[event][customer] = avitime else: total_time = events[event][customer] total_time += avitime events[event][customer] = total_time last_customer = customer events[event][customer] = human_time(events[event][customer]) df_events = pd.DataFrame(events) df.to_csv('event_track_times.csv') main()and here is the output I get from this: C:\Users\rob.kinsey\AppData\Local\Continuum\Anaconda3) c:\Users\rob.kinsey\Pro ramming>python test_single.py BARCELONA 03:52:42 02:38:31 03:21:02 00:16:35 00:59:00 00:17:45 01:31:42 03:03:03 03:16:43 01:08:03 01:59:54 00:09:03 COTA 04:38:42 02:42:34 sys:1: DtypeWarning: Columns (0) have mixed types. Specify dtype option on impo t or set low_memory=False. 04:01:13 01:19:47 03:09:31 02:37:32 03:37:34 02:14:42 04:53:01 LAGUNA_SECA 01:09:10 01:34:31 01:49:27 03:05:34 02:39:03 01:48:14 SILVERSTONE 04:39:31 01:52:21 02:53:42 02:10:44 02:11:17 02:37:11 01:19:12 04:32:21 05:06:43 SPA Traceback (most recent call last): File "test_single.py", line 56, in <module> main() File "test_single.py", line 41, in main events[event][last_customer] = human_time(events[event][last_customer]) File "test_single.py", line 23, in human_timeusing isnan gave me these errors? if isnan(avitime): (C:\Users\rob.kinsey\AppData\Local\Continuum\Anaconda3) c:\Users\rob.kinsey\Prog ramming>python test_single.py Traceback (most recent call last): File "test_single.py", line 57, in <module> main() File "test_single.py", line 34, in main if math.isnan(avitime): TypeError: a float is requiredThen using: if isnan(float(avitime) (C:\Users\rob.kinsey\AppData\Local\Continuum\Anaconda3) c:\Users\rob.kinsey\Prog ramming>python test_single.py Traceback (most recent call last): File "test_single.py", line 57, in <module> main() File "test_single.py", line 34, in main if math.isnan(float(avitime)): TypeError: float() argument must be a string or a number, not 'NoneType' RE: How to get past NaN value - micseydel - Dec-13-2016 It seems like avitime is None, not NaN or another float. RE: How to get past NaN value - iFunKtion - Dec-13-2016 if avitime == Nonegives exactly the same output. I just need to skip the whole file if the value is NaN on this particular occasion, though that is proving very difficult as I can't identify the file causing the probem, as they all open in a text file, some are empty or zero value on that element or they have the expected float, so it is difficult to understand what it is actually tripping up on, that's why I just want to completely skip the file is nan is present, but I can't find an answer. RE: How to get past NaN value - iFunKtion - Dec-14-2016 Has no one ever had this problem before? RE: How to get past NaN value - micseydel - Dec-14-2016 It's really tough to help without a snippet. Hard-code the value(s) that cause the problem, and provide a runnable snippet of code that reproduces the error you're seeing; it should be no more than 10 lines, and 5 is probably sufficient. In this specific case, I'm not clear on whether you're checking for both NaN and None or just one of them. Not having seen your most recent code, I can only say that you probably have to code for both, and you specifically have to check for None first since checking for NaN fails if it's None. And no, we probably haven't had this problem before, which is a common thing in programming and represents a decent amount of this forum :) RE: How to get past NaN value - iFunKtion - Jan-26-2017 @micseydel - Ok, yes I see your point. There is one point that you clarified that is very important to know, and I did not up until this point, and that is your last point, that this issue probably has not been seen before. I was always under the impression that most issues have been seen at some point, but this is an assumption, it's good to get clarification that often with programming, some issues are actually new, possibly because newbies like myself unlikely code things properly, there are an awful lot of docs out there to read, and most of them are useless to the complete newbie, I am now in the position where documentation is just starting to be useful, and I am 15 months into my programming career, it's a much steeper learning curve than I expected. Again, thank you, that last bit of information is very valuable to new programmers as posts that get no replies can leave one a little despondent and not knowing where to go to solve the problem. Have some more reputation. |