Python Forum
how can I improve the code to get it faster? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: how can I improve the code to get it faster? (/thread-24455.html)

how can I improve the code to get it faster? - aquerci - Feb-15-2020

Hi guys,

my script must read a text file and produce an output. it works quite well but a piece of his code is very slow (it because of some instructions at the beginning of my scrtipt). they are like the example below and take around 30 seconds to read a text file composed by around 300000 lines:

x = ""
y = ""
..
..

for line in text:
    list = line.split()
    if "xxxx" in line and "t" in list[5] and "z" not in list[5]
        x = x + line
    elif "xxxx" in line and "a" not in list[7] and "b" not in list[8] and " hola " not in line:
        y = y + line
    elif..
when I have many "not in" or "in" instructions for the same string, I tried to use this custom function, but unfortunately it doesn't solve the time issue:
def find_all_strings(text, strings_to_find, in_or_not_in):
    if in_or_not_in == "in":
        if all (string in text for string in strings_to_find):
            return True
        else:
            return False
    elif in_or_not_in == "not_in":
        if all (string not in text for string in strings_to_find):
            return True
        else:
            return False
is there a way to improve the code? my goal is implement a kind of "grep" and "grep -v" commands in the text document to take from it different configuration lines.

RE: how can I improve the code to get it faster? - Gribouillis - Feb-15-2020

Adding many pieces to a string as in x = x + line is slow. It is much faster to append to a list like so
x = []
...
    x.append(line)
...

x = ''.join(x) # <-- join the pieces at the end.


RE: how can I improve the code to get it faster? - aquerci - Feb-15-2020

thanks Gribouillis!! it works! the study books don't teach you this powerful and simple concepts. thank you again!