Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
read write
#1
def some_condition(line):
    return "ddd" in line

with open('testin.csv') as infile:
    for line in infile:
        if some_condition(line):
            donotline=line
        else:
            f = open('testout.csv','a+')
            f.write(line)
I have been trying to read from testin and write to testout.csv with the above code. The code runs but take approimatley 5 mins to read through 160000 lines. Is there any means to speed it up.
thanks
nuncio
Reply
#2
Don't open a new output file every time through the loop. Just open it once before the loop starts.
Reply
#3
Try this perhaps
import itertools as itt

def some_condition(line):
    return "ddd" in line

open('testout.csv', 'w').writelines(itt.filterfalse(condition, open('testin.csv')))
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python read PDF Statement and write it into excel mg24 1 972 Sep-22-2024, 11:42 AM
Last Post: Pedroski55
  Delete file with read-only permission, but write permission to parent folder cubei 6 25,437 Jun-01-2024, 07:22 AM
Last Post: Eleanorreo
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 3,840 Nov-09-2023, 10:56 AM
Last Post: mg24
Question Special Characters read-write Prisonfeed 1 1,438 Sep-17-2023, 08:26 PM
Last Post: Gribouillis
  How do I read and write a binary file in Python? blackears 6 25,061 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Read text file, modify it then write back Pavel_47 5 4,630 Feb-18-2023, 02:49 PM
Last Post: deanhystad
  how to read txt file, and write into excel with multiply sheet jacklee26 14 16,940 Jan-21-2023, 06:57 AM
Last Post: jacklee26
  Read JSON via API and write to SQL database TecInfo 5 4,834 Aug-09-2022, 04:44 PM
Last Post: TecInfo
  Write and read back data Aggie64 6 3,069 Apr-18-2022, 03:23 PM
Last Post: bowlofred
  UART Serial Read & Write to MP3 Player Doesn't Work bill_z 15 8,906 Jul-17-2021, 04:19 PM
Last Post: bill_z

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020