Python Forum
Need help with threads in for loop
Thread Rating:
  • 3 Vote(s) - 4 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Need help with threads in for loop
#1
Hello Friends,

I am using a nested loop first loop iterate for 50,000 times and second loop iterate for 500 times if i use normal approach
i.e.
for i in range(0,50000):
    for j in range(0,250)
        print(i,j)
It is taking huge time so can any one tell me how can i execute this nested for loop faster? can you please share the code as well

i am using this code but not very much fast.


import time

from threading import Thread

class GetMyThread(Thread):
    def __init__(self, start, end, start2, end2):
        self.ix = start 
        self.iy= end
        self.jx = start2
        self.jy = end2
        super(GetMyThread, self).__init__()
        for i in range(self.ix, self.iy):
            for j in range(0, len(overallKeywordsFrequency)):
                    if(overallKeywordsFrequency[j] in (df.iloc[i]['Keywords'].lower().strip())):
                        matrix[i,j] = 1
       
        
        
                

def get_responses():
    start = time.time()
    threads = []
    num_splits = 50
    split_size = len(df)//num_splits
    split_size2 = len(overallKeywordsFrequency)//num_splits
    for i in  range(num_splits):
        start = i * split_size
        start2 = i * split_size2
        end = len(df) if i+1 == num_splits else (i+1) * split_size
        end2 = len(overallKeywordsFrequency) if i+1 == num_splits else (i+1) * split_size2
        t = GetMyThread(start, end, start2, end2)
        threads.append(t)
        t.start()
    for t in threads:
        t.join()

get_responses()
Reply
#2
I have the suspicion that your code is taking an exceedingly long time due to a bug. The GetMyThread.__init__() method is running the matrix loop. Properly, that loop should be in the GetMyThread.run() method instead. I say this because you later call the thread.join() method and that method listens for the termination of the thread.run() method according to the documentation. If the thread.run() method isn't being used, thread.join() should not work. In turn, that means that you have a thread running and the program isn't stopping to collect its data.

Also, I don't see any mechanism for collecting the data from the thread.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Trying to separate a loop across multiple threads stylingpat 0 1,651 May-05-2021, 05:21 PM
Last Post: stylingpat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020