Python Forum
Need help with threads in for loop - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/Forum-Python-Coding)
+--- Forum: General Coding Help (https://python-forum.io/Forum-General-Coding-Help)
+--- Thread: Need help with threads in for loop (/Thread-Need-help-with-threads-in-for-loop)



Need help with threads in for loop - yestoprince - Jan-11-2019

Hello Friends,

I am using a nested loop first loop iterate for 50,000 times and second loop iterate for 500 times if i use normal approach
i.e.
for i in range(0,50000):
    for j in range(0,250)
        print(i,j)
It is taking huge time so can any one tell me how can i execute this nested for loop faster? can you please share the code as well

i am using this code but not very much fast.


import time

from threading import Thread

class GetMyThread(Thread):
    def __init__(self, start, end, start2, end2):
        self.ix = start 
        self.iy= end
        self.jx = start2
        self.jy = end2
        super(GetMyThread, self).__init__()
        for i in range(self.ix, self.iy):
            for j in range(0, len(overallKeywordsFrequency)):
                    if(overallKeywordsFrequency[j] in (df.iloc[i]['Keywords'].lower().strip())):
                        matrix[i,j] = 1
       
        
        
                

def get_responses():
    start = time.time()
    threads = []
    num_splits = 50
    split_size = len(df)//num_splits
    split_size2 = len(overallKeywordsFrequency)//num_splits
    for i in  range(num_splits):
        start = i * split_size
        start2 = i * split_size2
        end = len(df) if i+1 == num_splits else (i+1) * split_size
        end2 = len(overallKeywordsFrequency) if i+1 == num_splits else (i+1) * split_size2
        t = GetMyThread(start, end, start2, end2)
        threads.append(t)
        t.start()
    for t in threads:
        t.join()

get_responses()



RE: Need help with threads in for loop - stullis - Jan-11-2019

I have the suspicion that your code is taking an exceedingly long time due to a bug. The GetMyThread.__init__() method is running the matrix loop. Properly, that loop should be in the GetMyThread.run() method instead. I say this because you later call the thread.join() method and that method listens for the termination of the thread.run() method according to the documentation. If the thread.run() method isn't being used, thread.join() should not work. In turn, that means that you have a thread running and the program isn't stopping to collect its data.

Also, I don't see any mechanism for collecting the data from the thread.