Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
read write
#1
def some_condition(line):
    return "ddd" in line

with open('testin.csv') as infile:
    for line in infile:
        if some_condition(line):
            donotline=line
        else:
            f = open('testout.csv','a+')
            f.write(line)
I have been trying to read from testin and write to testout.csv with the above code. The code runs but take approimatley 5 mins to read through 160000 lines. Is there any means to speed it up.
thanks
nuncio
Reply
#2
Don't open a new output file every time through the loop. Just open it once before the loop starts.
Reply
#3
Try this perhaps
import itertools as itt

def some_condition(line):
    return "ddd" in line

open('testout.csv', 'w').writelines(itt.filterfalse(condition, open('testin.csv')))
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,314 Nov-09-2023, 10:56 AM
Last Post: mg24
Question Special Characters read-write Prisonfeed 1 582 Sep-17-2023, 08:26 PM
Last Post: Gribouillis
  How do I read and write a binary file in Python? blackears 6 6,016 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Read text file, modify it then write back Pavel_47 5 1,501 Feb-18-2023, 02:49 PM
Last Post: deanhystad
  how to read txt file, and write into excel with multiply sheet jacklee26 14 9,523 Jan-21-2023, 06:57 AM
Last Post: jacklee26
  Read JSON via API and write to SQL database TecInfo 5 2,114 Aug-09-2022, 04:44 PM
Last Post: TecInfo
  Write and read back data Aggie64 6 1,812 Apr-18-2022, 03:23 PM
Last Post: bowlofred
  UART Serial Read & Write to MP3 Player Doesn't Work bill_z 15 5,647 Jul-17-2021, 04:19 PM
Last Post: bill_z
  Read and write active Excel file euras 4 3,381 Jun-29-2021, 11:16 PM
Last Post: Pedroski55
  Read file, reformat and write new file bryanmartin113 1 2,193 Jun-08-2021, 09:27 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020