Python Forum
how can I improve the code to get it faster?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how can I improve the code to get it faster?
#1
Hi guys,

my script must read a text file and produce an output. it works quite well but a piece of his code is very slow (it because of some instructions at the beginning of my scrtipt). they are like the example below and take around 30 seconds to read a text file composed by around 300000 lines:

x = ""
y = ""
..
..

for line in text:
    list = line.split()
    if "xxxx" in line and "t" in list[5] and "z" not in list[5]
        x = x + line
    elif "xxxx" in line and "a" not in list[7] and "b" not in list[8] and " hola " not in line:
        y = y + line
    elif..
when I have many "not in" or "in" instructions for the same string, I tried to use this custom function, but unfortunately it doesn't solve the time issue:
def find_all_strings(text, strings_to_find, in_or_not_in):
    if in_or_not_in == "in":
        if all (string in text for string in strings_to_find):
            return True
        else:
            return False
    elif in_or_not_in == "not_in":
        if all (string not in text for string in strings_to_find):
            return True
        else:
            return False
is there a way to improve the code? my goal is implement a kind of "grep" and "grep -v" commands in the text document to take from it different configuration lines.
Reply
#2
Adding many pieces to a string as in x = x + line is slow. It is much faster to append to a list like so
x = []
...
    x.append(line)
...

x = ''.join(x) # <-- join the pieces at the end.
Reply
#3
thanks Gribouillis!! it works! the study books don't teach you this powerful and simple concepts. thank you again!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  faster code for my code kucingkembar 19 3,114 Aug-09-2022, 09:48 AM
Last Post: DPaul
  (OpenCV) Help to improve code for object detection and finding center of it saoko 0 1,159 May-14-2022, 05:34 PM
Last Post: saoko
  How can I improve this piece of code? aquerci 3 2,172 Nov-17-2019, 10:57 AM
Last Post: Gribouillis
  Help improve code efficiency benbrown03 9 4,253 Feb-20-2019, 03:45 PM
Last Post: ichabod801
  How to improve the quality of my code? grobattac37 3 2,449 Jan-25-2019, 06:17 PM
Last Post: ichabod801
  Another working code, help required for faster multithreading execution anna 0 2,235 Feb-09-2018, 03:26 AM
Last Post: anna
  Help required for faster execution of working code anna 2 3,105 Feb-09-2018, 03:00 AM
Last Post: anna
  How do I make this code run faster? DontHurtMe 0 2,393 Nov-04-2017, 12:12 PM
Last Post: DontHurtMe

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020