Need help with threads in for loop - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: Need help with threads in for loop (/thread-15286.html) |
Need help with threads in for loop - yestoprince - Jan-11-2019 Hello Friends, I am using a nested loop first loop iterate for 50,000 times and second loop iterate for 500 times if i use normal approach i.e. for i in range(0,50000): for j in range(0,250) print(i,j)It is taking huge time so can any one tell me how can i execute this nested for loop faster? can you please share the code as well i am using this code but not very much fast. import time from threading import Thread class GetMyThread(Thread): def __init__(self, start, end, start2, end2): self.ix = start self.iy= end self.jx = start2 self.jy = end2 super(GetMyThread, self).__init__() for i in range(self.ix, self.iy): for j in range(0, len(overallKeywordsFrequency)): if(overallKeywordsFrequency[j] in (df.iloc[i]['Keywords'].lower().strip())): matrix[i,j] = 1 def get_responses(): start = time.time() threads = [] num_splits = 50 split_size = len(df)//num_splits split_size2 = len(overallKeywordsFrequency)//num_splits for i in range(num_splits): start = i * split_size start2 = i * split_size2 end = len(df) if i+1 == num_splits else (i+1) * split_size end2 = len(overallKeywordsFrequency) if i+1 == num_splits else (i+1) * split_size2 t = GetMyThread(start, end, start2, end2) threads.append(t) t.start() for t in threads: t.join() get_responses() RE: Need help with threads in for loop - stullis - Jan-11-2019 I have the suspicion that your code is taking an exceedingly long time due to a bug. The GetMyThread.__init__() method is running the matrix loop. Properly, that loop should be in the GetMyThread.run() method instead. I say this because you later call the thread.join() method and that method listens for the termination of the thread.run() method according to the documentation. If the thread.run() method isn't being used, thread.join() should not work. In turn, that means that you have a thread running and the program isn't stopping to collect its data. Also, I don't see any mechanism for collecting the data from the thread. |