Python Forum
Trying to delete rows above a specific datetime value
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Trying to delete rows above a specific datetime value
#18
Well i spoke to fast, it was working fine in my test script, but now that i implemented the changes above to my actual script, im getting the datetime column at the end even though i have the df.drop before writing the new csv file..

How can that happen? What is wrong with this now? The replacing works, the datetime filter works, but when its written to the new text file, the temp column df['DT'] is appended to the end of the 3 original columns.

Unless i opened an old text file this morning when i checked, i thought this was already working..

# THIS OPENS THE NEWLY CLEAN DATA IN ORDER TO REMOVE OLD RECORDS
df = pd.read_csv("Dates.txt", usecols=range(3), names=["Date", "Time", "Comment"])

df['Comment'] = [re.sub(r'(?:^|\W)Someone Else:(?:$|\W)','Dan: ', str(x)) for x in df['Comment']]
df['Comment'] = [re.sub(r'(?:^|\W)Elizabeth:(?:$|\W)','Ross: ', str(x)) for x in df['Comment']]

# THIS CONVERTS THE DATE COLUMN INTO A DATETIME FORMAT
df['DT'] = pd.to_datetime(df['Date'] + ' ' + df['Time'])

# HERE YOU NEED TO PROVIDE THE DATE YOU WANT TO KEEP GOING FORWARD
mask = (df['DT'] > '2022-05-02  9:32 AM')

# THIS RETURNS ALL ROWS GREATER THAN THE DATE PROVIDED ABOVE
df = df.loc[mask]

# DROP TEMP COLUMN BEFORE WRITING CSV
df['DT'].drop

# THIS IS THE FILTERED DATA RESULTS TO IMPORT INTO EXCEL
df.to_csv(r'C:\Users\mynewfile.txt', header=None, index=None, mode='a')
Reply


Messages In This Thread
RE: Trying to delete rows above a specific datetime value - by cubangt - May-09-2022, 08:19 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  delete specific row of entries jacksfrustration 3 460 Feb-13-2024, 11:13 PM
Last Post: deanhystad
  Reading Specific Rows In a CSV File finndude 3 1,042 Dec-13-2022, 03:19 PM
Last Post: finndude
  Pymysql delete specific rows in tableview stsxbel 2 1,132 Aug-18-2022, 09:50 AM
Last Post: ibreeden
  The code I have written removes the desired number of rows, but wrong rows Jdesi1983 0 1,660 Dec-08-2021, 04:42 AM
Last Post: Jdesi1983
Question [SOLVED] Delete specific characters from string lines EnfantNicolas 4 2,306 Oct-21-2021, 11:28 AM
Last Post: EnfantNicolas
  openpyxl incorrect delete rows VladislavM 6 4,263 Jul-19-2021, 08:54 AM
Last Post: VladislavM
  Pandas DataFrame combine rows by column value, where Date Rows are NULL rhat398 0 2,160 May-04-2021, 10:51 PM
Last Post: rhat398
  Indexing [::-1] to Reverse ALL 2D Array Rows, ALL 3D, 4D Array Columns & Rows Python Jeremy7 8 7,371 Mar-02-2021, 01:54 AM
Last Post: Jeremy7
  How to sort rows based on specific order Mekala 3 2,274 Jul-31-2020, 01:01 AM
Last Post: bowlofred
  How to extract specific rows and columns from a text file with Python Farhan 0 3,440 Mar-25-2020, 09:18 PM
Last Post: Farhan

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020