Python Forum

(Apr-24-2017, 04:39 PM)nilamo Wrote: [ -> ]

(Apr-24-2017, 04:30 PM)smw10c Wrote: [ -> ]Additionally, the code above did not work. It states that "name NaN is not defined".

You should probably share your entire code then, instead of a small snippet which we all knew wouldn't work :P

Haha okay!

dict3={'initial cost': ['$75000.00','$0.00','$30000.00','$1500.00','$19500.00'],'Total Est. Fee':['$986.00','$1144.00','$522.50', '$225.00', '$389.50']}
df3=pd.DataFrame(dict3)
df3['Total Est. Fee']=df3['Total Est. Fee'].str.split('$')
df3['Total Est. Fee']=df3['Total Est. Fee'].str.get(1)
df3['initial cost']=df3['initial cost'].str.split('$')
df3['initial cost']=df3['initial cost'].str.get(1)
df3['Total Est. Fee']=pd.to_numeric(df3['Total Est. Fee'],errors='coerce')
df3['diff']=df3['initial cost']-df3['Total Est. Fee']
df3.diff[df3.diff < 0] = NaN

For some reason the second to last line of code isn't running anymore even though it ran earlier...

Try to run/think over nilamo's point 3 - its same for your code. In body of your loop you set X to reference "cell" content, and after that you rebind X to a new value (NaN) - but you change just X, not dataframe cell. Its similar to

item = [1]
x = item[0]
x = 10

- if you check item list, it would be still [1] - unchanged.

(Apr-24-2017, 04:47 PM)zivoni Wrote: [ -> ]Try to run/think over nilamo's point 3 - its same for your code. In body of your loop you set X to reference "cell" content, and after that you rebind X to a new value (NaN) - but you change just X, not dataframe cell. Its similar to

item = [1]
x = item[0]
x = 10

- if you check item list, it would be still [1] - unchanged.

I understand what you are saying. I just don't get how I'd fix this in the loop above. Thank you for the further explanation. I would really appreciate if you showed me how to do it in the way I was trying so I can better understand Python logic.

(Apr-24-2017, 04:30 PM)smw10c Wrote: [ -> ]Additionally, the code above did not work. It states that "name NaN is not defined".

You have used NaN in your code - that was reason why i supposed that you imported it from numpy.

(Apr-24-2017, 04:41 PM)smw10cp Wrote: [ -> ]

dict3={'initial cost': ['$75000.00','$0.00','$30000.00','$1500.00','$19500.00'],'Total Est. Fee':['$986.00','$1144.00','$522.50', '$225.00', '$389.50']}
df3=pd.DataFrame(dict3)
df3['Total Est. Fee']=df3['Total Est. Fee'].str.split('$')
df3['Total Est. Fee']=df3['Total Est. Fee'].str.get(1)
df3['initial cost']=df3['initial cost'].str.split('$')
df3['initial cost']=df3['initial cost'].str.get(1)
df3['Total Est. Fee']=pd.to_numeric(df3['Total Est. Fee'],errors='coerce')
df3['diff']=df3['initial cost']-df3['Total Est. Fee']
df3.diff[df3.diff < 0] = NaN

For some reason the second to last line of code isn't running anymore even though it ran earlier...

You didnt convert df3['initial cost'] to a number. Morever you should modify last line to

df3['diff'][df3['diff'] < 0] = NaN

, as i did in my post (initially i forgot that .diff() is a dataframe method).

That code need imports for pandas and numpy - atleast

import pandas as pd
from numpy import NaN

likely you didnt run imports in your second run and that was why it crashed on second line.

(Apr-24-2017, 05:08 PM)zivoni Wrote: [ -> ]

(Apr-24-2017, 04:30 PM)smw10c Wrote: [ -> ]Additionally, the code above did not work. It states that "name NaN is not defined".

You have used NaN in your code - that was reason why i supposed that you imported it from numpy.

(Apr-24-2017, 04:41 PM)smw10cp Wrote: [ -> ]

dict3={'initial cost': ['$75000.00','$0.00','$30000.00','$1500.00','$19500.00'],'Total Est. Fee':['$986.00','$1144.00','$522.50', '$225.00', '$389.50']}
df3=pd.DataFrame(dict3)
df3['Total Est. Fee']=df3['Total Est. Fee'].str.split('$')
df3['Total Est. Fee']=df3['Total Est. Fee'].str.get(1)
df3['initial cost']=df3['initial cost'].str.split('$')
df3['initial cost']=df3['initial cost'].str.get(1)
df3['Total Est. Fee']=pd.to_numeric(df3['Total Est. Fee'],errors='coerce')
df3['diff']=df3['initial cost']-df3['Total Est. Fee']
df3.diff[df3.diff < 0] = NaN

For some reason the second to last line of code isn't running anymore even though it ran earlier...

You didnt convert df3['initial cost'] to a number. Morever you should modify last line to

df3['diff'][df3['diff'] < 0] = NaN

, as i did in my post (initially i forgot that .diff() is a dataframe method).

That code need imports for pandas and numpy - atleast

import pandas as pd
from numpy import NaN

likely you didnt run imports in your second run and that was why it crashed on second line.

I have run the requested imports. For some reason, df3['diff']=df3['initial cost']-df3['Total Est. Fee'] is no longer working, so I tested the code on a different column. It still is not working for some reason. Sorry if I am annoying you all :(

df3['initial cost'][df3['initial cost'] < 0] = NaN

As both snippsat and nilamo said before, you make it harder by not posting informations about your code and/or errors. Its hard to help when you just say "is not working", neither its helping when you are working with pandas without knowing that you are working with pandas ...

Your code with changes suggested in my previous post works fine for me:

Output:
In [1]: import pandas as pd
   ...: from numpy import NaN
   ...: dict3={'initial cost': ['$75000.00','$0.00','$30000.00','$1500.00','$19500.00'],'Total Est. Fee':['$986.00','$1144.00','$522.50', '$
   ...: 225.00', '$389.50']}
   ...: df3=pd.DataFrame(dict3)
   ...: df3['Total Est. Fee']=df3['Total Est. Fee'].str.split('$')
   ...: df3['Total Est. Fee']=df3['Total Est. Fee'].str.get(1)
   ...: df3['initial cost']=df3['initial cost'].str.split('$')
   ...: df3['initial cost']=df3['initial cost'].str.get(1)
   ...: df3['Total Est. Fee']=pd.to_numeric(df3['Total Est. Fee'],errors='coerce')
   ...: df3['initial cost'] = pd.to_numeric(df3['initial cost'], errors='coerce')
   ...: df3['diff']=df3['initial cost']-df3['Total Est. Fee']
   ...: df3['diff'][df3['diff'] < 0] = NaN
   ...: 

In [2]: df3
Out[2]: 
   Total Est. Fee  initial cost     diff
0           986.0       75000.0  74014.0
1          1144.0           0.0      NaN
2           522.5       30000.0  29477.5
3           225.0        1500.0   1275.0
4           389.5       19500.0  19110.5

For the first bit of code where I subtract initial cost from Total estimated fee this is the error I get:

TypeError: ufunc 'subtract' did not contain a loop with signature matching types dtype('<U32') dtype('<U32') dtype('<U32')

When I try df3['initial cost'][df3['initial cost'] < 0] = NaN, the error I get is:

TypeError: '<' not supported between instances of 'str' and 'int'

Sorry about not posting my errors. I'll start posting them from now on.

---

(Apr-24-2017, 05:35 PM)zivoni Wrote: [ -> ]As both snippsat and nilamo said before, you make it harder by not posting informations about your code and/or errors. Its hard to help when you just say "is not working", neither its helping when you are working with pandas without knowing that you are working with pandas ...

Your code with changes suggested in my previous post works fine for me:

Output:
In [1]: import pandas as pd
   ...: from numpy import NaN
   ...: dict3={'initial cost': ['$75000.00','$0.00','$30000.00','$1500.00','$19500.00'],'Total Est. Fee':['$986.00','$1144.00','$522.50', '$
   ...: 225.00', '$389.50']}
   ...: df3=pd.DataFrame(dict3)
   ...: df3['Total Est. Fee']=df3['Total Est. Fee'].str.split('$')
   ...: df3['Total Est. Fee']=df3['Total Est. Fee'].str.get(1)
   ...: df3['initial cost']=df3['initial cost'].str.split('$')
   ...: df3['initial cost']=df3['initial cost'].str.get(1)
   ...: df3['Total Est. Fee']=pd.to_numeric(df3['Total Est. Fee'],errors='coerce')
   ...: df3['initial cost'] = pd.to_numeric(df3['initial cost'], errors='coerce')
   ...: df3['diff']=df3['initial cost']-df3['Total Est. Fee']
   ...: df3['diff'][df3['diff'] < 0] = NaN
   ...: 

In [2]: df3
Out[2]: 
   Total Est. Fee  initial cost     diff
0           986.0       75000.0  74014.0
1          1144.0           0.0      NaN
2           522.5       30000.0  29477.5
3           225.0        1500.0   1275.0
4           389.5       19500.0  19110.5

For the first bit of code where I subtract initial cost from Total estimated fee this is the error I get:

TypeError: ufunc 'subtract' did not contain a loop with signature matching types dtype('<U32') dtype('<U32') dtype('<U32')

When I try df3['initial cost'][df3['initial cost'] < 0] = NaN, the error I get is:

TypeError: '<' not supported between instances of 'str' and 'int'

Sorry about not posting my errors. I'll start posting them from now on.

Also, sorry for not posting my response directly to your response. I accidentally made it as a separate entry.

(Apr-24-2017, 07:16 PM)smw10c Wrote: [ -> ]When I try df3['initial cost'][df3['initial cost'] < 0] = NaN, the error I get is:

TypeError: '<' not supported between instances of 'str' and 'int'

Did you convert df3['initial cost'] to numeric dtype, as was mentioned before?

Do not worry about not posting quote/response aligned, it takes some time to learn editor (but using Preview  Post button helps a lot)

(Apr-24-2017, 07:59 PM)zivoni Wrote: [ -> ]

(Apr-24-2017, 07:16 PM)smw10c Wrote: [ -> ]When I try df3['initial cost'][df3['initial cost'] < 0] = NaN, the error I get is: TypeError: '<' not supported between instances of 'str' and 'int'

Did you convert df3['initial cost'] to numeric dtype, as was mentioned before? Do not worry about not posting quote/response aligned, it takes some time to learn editor (but using Preview Post button helps a lot)

Thank you! That was the issue. I forgot to convert it to numeric type. Thanks again for bearing with me.