Python Forum

I would like to remove specific values from dataframe.

For example, my dataset

Output:union1
17    215 Marsiling                
18    653 Woodlands                
19    101 Tampines                 
20    36 Mt Vernon                 
21    408 Fajar Rd                 
22    158 Tampines                 
23    8 Kim Tian                   
24    704 Choa Chu Kang (BR)       
25    21 Sophia Hills              
26    90 Punggol Central           
27    46 Lengkong Tujoh            
28    9 Jln Mata Ayer (茶无糖)        
29    130 Bishan St                
30    12 Choa Chu Kang             
31    106A Depot Rd                
32    266D Punggol Way             
33    81 Telok Kurau (BR) (VIP)    
34    216C Compassvale             
35    130 Rivervale St             
36    222A Sumang                  
37    NaN                          
38    NaN                          
39    NO PASTING BEYOND THIS POINT 
17    821 Woodlands                
18    489C Tampines

show more (open the raw output data in a text editor) ...

Output:35    NaN                          
36    NaN                          
37    NaN                          
38    NaN                          
39    NaN                          
dtype: object

When I run command

df.drop(df.index[df['union1'] == 'NaN'], inplace = True)

Error:KeyError: 'union1'
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2645             try:
-> 2646                 return self._engine.get_loc(key)
   2647             except KeyError:

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'union1'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-312-f8a16c515c16> in <module>
----> 1 df.drop(df.index[df['union1'] == 'NaN'], inplace = True)

~\anaconda3\lib\site-packages\pandas\core\frame.py in __getitem__(self, key)
   2798             if self.columns.nlevels > 1:
   2799                 return self._getitem_multilevel(key)
-> 2800             indexer = self.columns.get_loc(key)
   2801             if is_integer(indexer):
   2802                 indexer = [indexer]

~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2646                 return self._engine.get_loc(key)
   2647             except KeyError:
-> 2648                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2649         indexer = self.get_indexer([key], method=method, tolerance=tolerance)
   2650         if indexer.ndim > 1 or indexer.size > 1:

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'union1'

Anybody can help?

jonah88888