Python Forum
How to get past NaN value
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to get past NaN value
#1
Hi there,

I am trying to do a simple test for a NaN value in python3 pandas, but I can't get anything to work. This is what I want to do, x is a float of value anything up to 1396253.0
if x == nan:
    continue
else
    rest of program...
I can't find any sytax for it, this itself gives a syntax error, I have seen also isnan() and can't get that working either. Is there a simple solution to this, as it has been cropping up a lot. A null value would have been much easier to deal with.
Reply
#2
In general, please show a real snippet of code you've tried that reproduces the problem. In this case, "rest of program..." is clearly a syntax error, so we can't tell what might be going on when you say that; if I put that aside, I'd expect a NameError, not a SyntaxError.

Here's a full session where isnan works for me
Output:
import math >>> math.isnan(0) False >>> math.isnan(float('inf')) False >>> math.isnan(float('nan')) True
Reply
#3
Ok, here is the code, I have tried to use the isnan, but it just throws back a TypeError:
#!/usr/bin/env python3

import glob
import math
import pandas as pd
import numpy as np

def get_avitime(vbo):
    try:
        df = pd.read_csv(vbo,
                         delim_whitespace=True,
                         header=90)
        row = next(df.iterrows())
        t = df.tail(2).avitime.values[0]
        return t
    except:
        pass


def human_time(seconds):
        secs = seconds/1000
        mins, secs = divmod(secs, 60)
        hours, mins = divmod(mins, 60)
        return '%02d:%02d:%02d' % (hours, mins, secs)


def main():
    path = 'Z:\\VBox_Backup\\**\\*.vbo'
    events = {}
    customers = {}

    for vbo_path in glob.glob(path, recursive=True):
        path_list = vbo_path.split('\\')
        event = path_list[2].upper()
        customer = path_list[3].title()
        avitime = get_avitime(vbo_path)
        if not avitime:             # this is where I tried      if math.isnan(float(avitime)):
            continue
        else:
            if event not in events:
                events[event] = {customer:avitime}
                print(event)
            elif customer not in events[event]:
                events[event][last_customer] = human_time(events[event][last_customer])
                print(last_customer, events[event][last_customer])
#                print(events[event][last_customer])
                events[event][customer] = avitime
            else:
                total_time = events[event][customer]
                total_time += avitime
                events[event][customer] = total_time
        last_customer = customer

    events[event][customer] = human_time(events[event][customer])
    df_events = pd.DataFrame(events)
    df.to_csv('event_track_times.csv')
    
main()
and here is the output I get from this:

C:\Users\rob.kinsey\AppData\Local\Continuum\Anaconda3) c:\Users\rob.kinsey\Pro
ramming>python test_single.py
BARCELONA
03:52:42
02:38:31
03:21:02
00:16:35
00:59:00
00:17:45
01:31:42
03:03:03
03:16:43
01:08:03
01:59:54
00:09:03
COTA
04:38:42
02:42:34
sys:1: DtypeWarning: Columns (0) have mixed types. Specify dtype option on impo
t or set low_memory=False.
04:01:13
01:19:47
03:09:31
02:37:32
03:37:34
02:14:42
04:53:01
LAGUNA_SECA
01:09:10
01:34:31
01:49:27
03:05:34
02:39:03
01:48:14
SILVERSTONE
04:39:31
01:52:21
02:53:42
02:10:44
02:11:17
02:37:11
01:19:12
04:32:21
05:06:43
SPA
Traceback (most recent call last):
  File "test_single.py", line 56, in <module>
    main()
  File "test_single.py", line 41, in main
    events[event][last_customer] = human_time(events[event][last_customer])
  File "test_single.py", line 23, in human_time
using isnan gave me these errors?
if isnan(avitime):


(C:\Users\rob.kinsey\AppData\Local\Continuum\Anaconda3) c:\Users\rob.kinsey\Prog
ramming>python test_single.py
Traceback (most recent call last):
  File "test_single.py", line 57, in <module>
    main()
  File "test_single.py", line 34, in main
    if math.isnan(avitime):
TypeError: a float is required
Then using:
if isnan(float(avitime)


(C:\Users\rob.kinsey\AppData\Local\Continuum\Anaconda3) c:\Users\rob.kinsey\Prog
ramming>python test_single.py
Traceback (most recent call last):
  File "test_single.py", line 57, in <module>
    main()
  File "test_single.py", line 34, in main
    if math.isnan(float(avitime)):
TypeError: float() argument must be a string or a number, not 'NoneType'
Reply
#4
It seems like avitime is None, not NaN or another float.
Reply
#5
if avitime == None
gives exactly the same output. I just need to skip the whole file if the value is NaN on this particular occasion, though that is proving very difficult as I can't identify the file causing the probem, as they all open in a text file, some are empty or zero value on that element or they have the expected float, so it is difficult to understand what it is actually tripping up on, that's why I just want to completely skip the file is nan is present, but I can't find an answer.
Reply
#6
Has no one ever had this problem before?
Reply
#7
It's really tough to help without a snippet. Hard-code the value(s) that cause the problem, and provide a runnable snippet of code that reproduces the error you're seeing; it should be no more than 10 lines, and 5 is probably sufficient.

In this specific case, I'm not clear on whether you're checking for both NaN and None or just one of them. Not having seen your most recent code, I can only say that you probably have to code for both, and you specifically have to check for None first since checking for NaN fails if it's None. And no, we probably haven't had this problem before, which is a common thing in programming and represents a decent amount of this forum :)
Reply
#8
@micseydel - Ok, yes I see your point. There is one point that you clarified that is very important to know, and I did not up until this point, and that is your last point, that this issue probably has not been seen before. I was always under the impression that most issues have been seen at some point, but this is an assumption, it's good to get clarification that often with programming, some issues are actually new, possibly because newbies like myself unlikely code things properly, there are an awful lot of docs out there to read, and most of them are useless to the complete newbie, I am now in the position where documentation is just starting to be useful, and I am 15 months into my programming career, it's a much steeper learning curve than I expected.

Again, thank you, that last bit of information is very valuable to new programmers as posts that get no replies can leave one a little despondent and not knowing where to go to solve the problem. Have some more reputation.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020