Python Forum
Remove if similar values available based on two columns
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Remove if similar values available based on two columns
#1
I want to remove the duplicates (keeping only the first occurrence) if similar values are found between two columns.

import pandas as pd
df1 = pd.DataFrame({'Customer':['John Tom', 'Harry', 'Simon', 'Tom'],
                    'Supplier':['Harry', 'John Tom', 'Harry', 'Simon']})
ie. I want to remove the second-row values as both values in that row were found previously in two columns in one row.

My expected result
Output:
Customer Supplier 0 John Tom Harry 1 Simon Harry 2 Tom Simon
Is there a way to remove such occurrences by considering them as duplicates?

appreciate it if someone can help
Reply
#2
import pandas as pd

df1 = pd.DataFrame({'Customer':['John Tom', 'Harry', 'Simon', 'Tom'],
    'Supplier':['Harry', 'John Tom', 'Harry', 'Simon']})

for idx in df1:
    eseries = pd.Series(df1[idx])
    print(f"df1[{idx}] unique = {eseries.unique()}")
results:
Output:
df1[Customer] unique = ['John Tom' 'Harry' 'Simon' 'Tom'] df1[Supplier] unique = ['Harry' 'John Tom' 'Simon']
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  remove duplicates from dicts with list values wardancer84 27 5,775 May-27-2024, 04:54 PM
Last Post: wardancer84
  Converting column of values into muliple columns of counts highland44 0 907 Feb-01-2024, 12:48 AM
Last Post: highland44
  unable to remove all elements from list based on a condition sg_python 3 1,711 Jan-27-2024, 04:03 PM
Last Post: deanhystad
  Remove some columns James_S 4 1,731 Dec-16-2023, 11:02 PM
Last Post: James_S
  Remove values for weekend in a panda series JaneTan 0 1,276 Dec-12-2022, 01:50 AM
Last Post: JaneTan
  Need to parse a list of boolean columns inside a list and return true values Python84 4 3,237 Jan-09-2022, 02:39 AM
Last Post: Python84
  Calculate next rows based on previous values of array divon 0 2,987 Nov-23-2021, 04:44 AM
Last Post: divon
  Sum the values in a pandas pivot table specific columns klllmmm 1 6,298 Nov-19-2021, 04:43 PM
Last Post: klllmmm
  How to remove a column or two columns in a correlation heatmap? lulu43366 3 7,639 Sep-30-2021, 03:47 PM
Last Post: lulu43366
  Remove Specific Columns when the number of columns is greater than a specific value CuriousOne 0 1,843 Sep-09-2021, 09:17 PM
Last Post: CuriousOne

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020