Python Forum

Python Forum > Python Coding > General Coding Help > how to start a seperate thread for every item in a list
Full Version: how to start a seperate thread for every item in a list
You're currently viewing a stripped down version of our content. View the full version with proper formatting.

lravikumarvsp

May-31-2018, 12:33 AM
Hi,

I want to create a separate thread for every item in a list.
But with my script, each thread is taking all the items in a list.
Please help.
import glob
import os
import csv
import executor
import concurrent.futures
import threading
from threading import Thread
import time
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
def process(item): 
	items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
	print "Sleeping 2 sec from thread %s \n" % item
	
	time.sleep(1)
	#i=len(items)
	print "Finished sleeping from thread %s \n" % item
	for item in items:
		try:
			base_path = '/ai2/data/dev/admin/inf/*{}*'
			search_path =  base_path.format(item)
			for f in glob.glob(search_path):
				print ("The directory path is:")
				print f

	
		except:
			print('error with pattern')
for item in items:
	t = Thread(target=process, args=(item,))
	t.start()
	t.join()
Output:
Sleeping 2 sec from thread ACM 

Finished sleeping from thread ACM 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread ACX 

Finished sleeping from thread ACX 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread AW 

Finished sleeping from thread AW 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread BC 

Finished sleeping from thread BC 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread XU0 

Finished sleeping from thread XU0 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread DRM 

Finished sleeping from thread DRM 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread DHD 

Finished sleeping from thread DHD 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread CR 

Finished sleeping from thread CR 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread GSK 

Finished sleeping from thread GSK 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread DMS 

Finished sleeping from thread DMS 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
Sleeping 2 sec from thread BLS 

Finished sleeping from thread BLS 

The directory path is:
/ai2/data/dev/admin/inf/inf_ACM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_ACX_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_AW_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BC_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DHD_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_CR_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_DMS_pvt
The directory path is:
/ai2/data/dev/admin/inf/inf_BLS_pvt
My required output should be like thread ACM must give the directory path containing ACM "/ai2/data/dev/admin/inf/inf_ACM_pvt"
Hi,

I got this by modifying the code as below
import glob
import os
import csv
import executor
import concurrent.futures
import threading
from threading import Thread
import time
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
def process(item): 
	items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
	print "Sleeping 2 sec from thread %s \n" % item
	
	time.sleep(1)
	#i=len(items)
	print "Finished sleeping from thread %s \n" % item
	
	try:
		base_path = '/ai2/data/dev/admin/inf/*{}*'
		search_path =  base_path.format(item)
		for f in glob.glob(search_path):
			print ("The directory path is:")
			print f

	
	except:
		print('error with pattern')
for item in items:
	t = Thread(target=process, args=(item,))
	t.start()
	t.join()
Now my requirement is I want to run 5 threads in parallel at a time and if running thread count is less than 5 I want to add another thread to it.
Please help

ThiefOfTime

May-31-2018, 08:07 AM
It is because you are creating a new list items in the process method and iterate over all of that items. Change the method to:
def process(item):
    print "Sleeping 2 sec from thread %s \n" % item
     
    time.sleep(1)
    #i=len(items)
    print "Finished sleeping from thread %s \n" % item
    try:
        base_path = '/ai2/data/dev/admin/inf/*{}*'
        search_path =  base_path.format(item)
        for f in glob.glob(search_path):
            print ("The directory path is:")
            print f
    except:
        print('error with pattern')
Python Forum > Python Coding > General Coding Help > how to start a seperate thread for every item in a list