Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
read write
#1
def some_condition(line):
    return "ddd" in line

with open('testin.csv') as infile:
    for line in infile:
        if some_condition(line):
            donotline=line
        else:
            f = open('testout.csv','a+')
            f.write(line)
I have been trying to read from testin and write to testout.csv with the above code. The code runs but take approimatley 5 mins to read through 160000 lines. Is there any means to speed it up.
thanks
nuncio
Reply
#2
Don't open a new output file every time through the loop. Just open it once before the loop starts.
Reply
#3
Try this perhaps
import itertools as itt

def some_condition(line):
    return "ddd" in line

open('testout.csv', 'w').writelines(itt.filterfalse(condition, open('testin.csv')))
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,525 Nov-09-2023, 10:56 AM
Last Post: mg24
Question Special Characters read-write Prisonfeed 1 648 Sep-17-2023, 08:26 PM
Last Post: Gribouillis
  How do I read and write a binary file in Python? blackears 6 6,872 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Read text file, modify it then write back Pavel_47 5 1,668 Feb-18-2023, 02:49 PM
Last Post: deanhystad
  how to read txt file, and write into excel with multiply sheet jacklee26 14 10,215 Jan-21-2023, 06:57 AM
Last Post: jacklee26
  Read JSON via API and write to SQL database TecInfo 5 2,239 Aug-09-2022, 04:44 PM
Last Post: TecInfo
  Write and read back data Aggie64 6 1,920 Apr-18-2022, 03:23 PM
Last Post: bowlofred
  UART Serial Read & Write to MP3 Player Doesn't Work bill_z 15 5,883 Jul-17-2021, 04:19 PM
Last Post: bill_z
  Read and write active Excel file euras 4 3,542 Jun-29-2021, 11:16 PM
Last Post: Pedroski55
  Read file, reformat and write new file bryanmartin113 1 2,238 Jun-08-2021, 09:27 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020