Script getting reindexing only valid error

[cubangt]

So this script has been running perfecting fine for well over a month if not longer, script has not been modified or changed in anyway. The script does read from a text file to build a dataframe using pandas, and that did change today. BUT the only thing that changed was additional records were added to the text file. Saved and then received this error when run.

Error:
  File c:\users\pinglistvariable_new.py:74
    merged_results = pd.concat(

  File ~\Miniconda3\lib\site-packages\pandas\util\_decorators.py:331 in wrapper
    return func(*args, **kwargs)

  File ~\Miniconda3\lib\site-packages\pandas\core\reshape\concat.py:381 in concat
    return op.get_result()

  File ~\Miniconda3\lib\site-packages\pandas\core\reshape\concat.py:612 in get_result
    indexers[ax] = obj_labels.get_indexer(new_labels)

  File ~\Miniconda3\lib\site-packages\pandas\core\indexes\base.py:3904 in get_indexer
    raise InvalidIndexError(self._requires_unique_msg)

InvalidIndexError: Reindexing only valid with uniquely valued Index objects

The line it complains about is this line:

merged_results = pd.concat((source.set_index("IP"), df.set_index(0)),axis=1).reset_index()

Logic around the above error line:

# WRITES THE COMPLETE LIST OF RESULTS TO THE DATAFRAME FOR SAVING
df = pd.DataFrame(executor.map(ping, list(scan_files())))
# ADDS A THIRD COLUMN WITH THE RUNDATE AND TIME TO THE DATAFRAME BEFORE SAVING
df[2] = dt

# MERGE SOURCE WITH RESULTS TO HAVE A RESULT FILE WITH STORE AND REGISTER INFORMATION
# DEBUGGING UNIQUE INDEX
WRON1 = source.index.is_unique     ==== when executed this returns "True"
WRON2 = df.index.is_unique            ==== when executed this returns "True"
merged_results = pd.concat((source.set_index("IP"), df.set_index(0)), axis=1).reset_index()

# adding header 
headerList = ['IP', 'Store', 'Register #', 'Status', 'Runtime']

# SAVES THE RESULTS TO A CSV FILE FOR THE EXCEL FILE DASHBOARD
# IP, 0 = UP 1 = DOWN,RUN DATETIME
merged_results.to_csv(parts[0] + '_ip_output_' + dtstring +'.csv',header=headerList, index=False, quoting=None)

I tested both the dataframes using : .index.is_unique and both came back as "true" which from what i read means that they both have unique indexes, is that correct? Did i read that correctly on other sites?

Now i did use notepad ++ to compare my old text file with the new text file and the only difference was the new records. BUT what bothers me is that, i was assuming it was the new text file, but i reverted the text file back to the previous file and ran the script again and it also failed.. The other kicker is that this script is automatically run twice a day and it ran successfully this morning during the first run, and between runs, i added about 50 more IP's addresses to the text file and let the automation run at 4pm where it failed with that above error.

Again, the script code has not been touched since 10/26 and same goes for the text file that was modified today, prior to modification, also 10/26

I dont have the code to post now, but can the above concat line be tweaked or replaced with something a little more reliable? I was reading that instead of concat, others use merge or join. But i i cant be changing the code now during the holiday, i need to find out why the change to the text file caused working code to break.

Any suggestions on things to check or validate are correct? Only thing that has changed since oct was the text file that the script uses to run. Script code was not and has not been changed(yet) need to try and resolve today.

[cubangt]

Disregard the above error.. it was an oversight on my end, the script is looking for a specific file name to process, and because i only renamed the orginal file and left it in the folder, when it would run, it was picking up both the new file and the original and creating a dataframe with double the records and causing the index issue because im using the "IP" as the index and there were duplicates.

The script successfully ran this morning and back in business.