Python Forum
Remove specific values from dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Remove specific values from dataframe
#1
I would like to remove specific values from dataframe.

For example, my dataset

Output:
union1 17 215 Marsiling 18 653 Woodlands 19 101 Tampines 20 36 Mt Vernon 21 408 Fajar Rd 22 158 Tampines 23 8 Kim Tian 24 704 Choa Chu Kang (BR) 25 21 Sophia Hills 26 90 Punggol Central 27 46 Lengkong Tujoh 28 9 Jln Mata Ayer (茶无糖) 29 130 Bishan St 30 12 Choa Chu Kang 31 106A Depot Rd 32 266D Punggol Way 33 81 Telok Kurau (BR) (VIP) 34 216C Compassvale 35 130 Rivervale St 36 222A Sumang 37 NaN 38 NaN 39 NO PASTING BEYOND THIS POINT 17 821 Woodlands 18 489C Tampines
show more (open the raw output data in a text editor) ...

Output:
35 NaN 36 NaN 37 NaN 38 NaN 39 NaN dtype: object
When I run command
df.drop(df.index[df['union1'] == 'NaN'], inplace = True)
Error:
KeyError: 'union1' --------------------------------------------------------------------------- KeyError Traceback (most recent call last) ~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance) 2645 try: -> 2646 return self._engine.get_loc(key) 2647 except KeyError: pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc() pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc() pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item() pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: 'union1' During handling of the above exception, another exception occurred: KeyError Traceback (most recent call last) <ipython-input-312-f8a16c515c16> in <module> ----> 1 df.drop(df.index[df['union1'] == 'NaN'], inplace = True) ~\anaconda3\lib\site-packages\pandas\core\frame.py in __getitem__(self, key) 2798 if self.columns.nlevels > 1: 2799 return self._getitem_multilevel(key) -> 2800 indexer = self.columns.get_loc(key) 2801 if is_integer(indexer): 2802 indexer = [indexer] ~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance) 2646 return self._engine.get_loc(key) 2647 except KeyError: -> 2648 return self._engine.get_loc(self._maybe_cast_indexer(key)) 2649 indexer = self.get_indexer([key], method=method, tolerance=tolerance) 2650 if indexer.ndim > 1 or indexer.size > 1: pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc() pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc() pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item() pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: 'union1'
Anybody can help?
Yoriz write Sep-24-2021, 06:07 AM:
Please post all code, output and errors (in their entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button.
buran write Sep-24-2021, 06:05 AM:
Please, use proper tags when post code, traceback, output, etc.
See BBcode help for more info.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  attempt to split values from within a dataframe column mbrown009 8 2,217 Apr-10-2023, 02:06 AM
Last Post: mbrown009
  update values in one dataframe based on another dataframe - Pandas iliasb 2 9,099 Aug-14-2021, 12:38 PM
Last Post: jefsummers
  Setting the x-axis to a specific column in a dataframe devansing 0 1,993 May-23-2021, 12:11 AM
Last Post: devansing
  How to search for specific string in Pandas dataframe Coding_Jam 1 2,415 Nov-02-2020, 09:35 AM
Last Post: PsyPy
  merging dataframe into another on a specific row danlin123 0 1,431 Sep-23-2020, 02:42 PM
Last Post: danlin123
  Assigning Column nunique values to another DataFrame column Pythonito 0 1,683 Jun-25-2020, 05:04 PM
Last Post: Pythonito
  How to create a dataframe from timestamp values MohanReddy 5 4,103 Mar-20-2019, 09:11 AM
Last Post: samsonite
  Referring to a specific element in Pandas Dataframe Helmi 2 3,167 Mar-17-2019, 09:12 PM
Last Post: Helmi
  Write specific rows from pandas dataframe to csv file pradeepkumarbe 3 5,426 Oct-18-2018, 09:33 PM
Last Post: volcano63
  Splitting values in column in a pandas dataframe based on a condition hey_arnold 1 4,139 Jul-24-2018, 02:18 PM
Last Post: hey_arnold

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020