Python Forum
Unable to delete duplicates in excel with Python - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Unable to delete duplicates in excel with Python (/thread-6127.html)



Unable to delete duplicates in excel with Python - tysondogerz - Nov-07-2017

I am trying to delete duplicates but the job just finishes with an exit code 0 and does not delete any duplicates.

I have attempted to do this with openpyxl for an excel as well as other methods (including csv though this deleted rows excessively).

The duplicates for the data always exist in Column F and I am desiring to delete the entire row B-I

Any ideas?

import openpyxl
wb1 = openpyxl.load_workbook('C:/dwad/SWWA.xlsx')
ws1 = wb1.active # keep naming convention consistent

values = []
for i in range(2,ws1.max_row+1):
  if ws1.cell(row=i,column=1).value in values:
    #pass
  #else:
    values.append(ws1.cell(row=i,column=1).value)

for value in values:
  ws1.append([value])

CSV:
with open('1.csv','r') as in_file, open('2.csv','w') as out_file:
    seen = set() # set for fast O(1) amortized lookup
    for line in in_file:
        if line not in seen: 
            seen.add(line)
            out_file.write(line)
[Image: cHzLyG]


RE: Unable to delete duplicates in excel with Python - tysondogerz - Nov-07-2017

Trying to set it for a set range... As it's only getting one row so far.. Likely have to iterate...

wb1 = openpyxl.load_workbook('C:/adw.xlsx')
ws = wb1.active
wb2 = openpyxl.Workbook()
ws2 = wb2.active

ws1 = wb1.active  # keep naming convention consistent

column = ['A:B']
values = []
for row in range(1, ws1.max_row):
    if ws1.cell(row=row, column=row).value in values:
        pass  # if already in list do nothing
    else:
        values.append(ws1.cell(row=row, column=row).value)

directory = 'C:/dwadwaddwad.csv'
with open(directory, 'a', newline='', encoding="utf-8") as outfile:
    for value in values:
        ws2.append([value])
        print(value)
wb2.save('C:/dwadwadaw.xlsx')

I don't think openpyxl supports this. It can be done with line in Vba though https://msdn.microsoft.com/en-us/vba/excel-vba/articles/range-removeduplicates-method-excel


RE: Unable to delete duplicates in excel with Python - tysondogerz - Nov-07-2017

Solved. I use about 400 lines of code. But it does the job if you just iterate through the same file 12 times.