how to start a seperate thread for every item in a list - lravikumarvsp - May-31-2018
Hi,
I want to create a separate thread for every item in a list.
But with my script, each thread is taking all the items in a list.
Please help.
import glob
import os
import csv
import executor
import concurrent.futures
import threading
from threading import Thread
import time
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
def process(item):
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
print "Sleeping 2 sec from thread %s \n" % item
time.sleep(1)
#i=len(items)
print "Finished sleeping from thread %s \n" % item
for item in items:
try:
base_path = '/ai2/data/dev/admin/inf/*{}*'
search_path = base_path.format(item)
for f in glob.glob(search_path):
print ("The directory path is:")
print f
except:
print('error with pattern')
for item in items:
t = Thread(target=process, args=(item,))
t.start()
t.join() Output: Sleeping 2 sec from thread ACM
Finished sleeping from thread ACM
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread ACX
Finished sleeping from thread ACX
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread AW
Finished sleeping from thread AW
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread BC
Finished sleeping from thread BC
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread XU0
Finished sleeping from thread XU0
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread DRM
Finished sleeping from thread DRM
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread DHD
Finished sleeping from thread DHD
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread CR
Finished sleeping from thread CR
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread GSK
Finished sleeping from thread GSK
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread DMS
Finished sleeping from thread DMS
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread BLS
Finished sleeping from thread BLS
The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
My required output should be like thread ACM must give the directory path containing ACM "/ai2/data/dev/admin/inf/inf_ACM_pvt"
Hi,
I got this by modifying the code as below
import glob
import os
import csv
import executor
import concurrent.futures
import threading
from threading import Thread
import time
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
def process(item):
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
print "Sleeping 2 sec from thread %s \n" % item
time.sleep(1)
#i=len(items)
print "Finished sleeping from thread %s \n" % item
try:
base_path = '/ai2/data/dev/admin/inf/*{}*'
search_path = base_path.format(item)
for f in glob.glob(search_path):
print ("The directory path is:")
print f
except:
print('error with pattern')
for item in items:
t = Thread(target=process, args=(item,))
t.start()
t.join() Now my requirement is I want to run 5 threads in parallel at a time and if running thread count is less than 5 I want to add another thread to it.
Please help
RE: how to start a seperate thread for every item in a list - ThiefOfTime - May-31-2018
It is because you are creating a new list items in the process method and iterate over all of that items. Change the method to:
def process(item):
print "Sleeping 2 sec from thread %s \n" % item
time.sleep(1)
#i=len(items)
print "Finished sleeping from thread %s \n" % item
try:
base_path = '/ai2/data/dev/admin/inf/*{}*'
search_path = base_path.format(item)
for f in glob.glob(search_path):
print ("The directory path is:")
print f
except:
print('error with pattern')
|