Python Forum
how to start a seperate thread for every item in a list
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to start a seperate thread for every item in a list
#1
Hi,

I want to create a separate thread for every item in a list.
But with my script, each thread is taking all the items in a list.
Please help.
import glob
import os
import csv
import executor
import concurrent.futures
import threading
from threading import Thread
import time
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
def process(item): 
	items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
	print "Sleeping 2 sec from thread %s \n" % item
	
	time.sleep(1)
	#i=len(items)
	print "Finished sleeping from thread %s \n" % item
	for item in items:
		try:
			base_path = '/ai2/data/dev/admin/inf/*{}*'
			search_path =  base_path.format(item)
			for f in glob.glob(search_path):
				print ("The directory path is:")
				print f

	
		except:
			print('error with pattern')
for item in items:
	t = Thread(target=process, args=(item,))
	t.start()
	t.join()
Output:
Sleeping 2 sec from thread ACM Finished sleeping from thread ACM The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread ACX Finished sleeping from thread ACX The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread AW Finished sleeping from thread AW The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread BC Finished sleeping from thread BC The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread XU0 Finished sleeping from thread XU0 The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread DRM Finished sleeping from thread DRM The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread DHD Finished sleeping from thread DHD The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread CR Finished sleeping from thread CR The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread GSK Finished sleeping from thread GSK The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread DMS Finished sleeping from thread DMS The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt Sleeping 2 sec from thread BLS Finished sleeping from thread BLS The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt The directory path is: /ai2/data/dev/admin/inf/inf_XU0_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DRM_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DHD_pvt The directory path is: /ai2/data/dev/admin/inf/inf_CR_pvt The directory path is: /ai2/data/dev/admin/inf/inf_GSK_pvt The directory path is: /ai2/data/dev/admin/inf/inf_DMS_pvt The directory path is: /ai2/data/dev/admin/inf/inf_BLS_pvt
My required output should be like thread ACM must give the directory path containing ACM "/ai2/data/dev/admin/inf/inf_ACM_pvt"

Hi,

I got this by modifying the code as below
import glob
import os
import csv
import executor
import concurrent.futures
import threading
from threading import Thread
import time
items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
def process(item): 
	items = items = ['ACM','ACX','AW','BC','XU0', 'DRM', 'DHD', 'CR', 'GSK', 'DMS', 'BLS']
	print "Sleeping 2 sec from thread %s \n" % item
	
	time.sleep(1)
	#i=len(items)
	print "Finished sleeping from thread %s \n" % item
	
	try:
		base_path = '/ai2/data/dev/admin/inf/*{}*'
		search_path =  base_path.format(item)
		for f in glob.glob(search_path):
			print ("The directory path is:")
			print f

	
	except:
		print('error with pattern')
for item in items:
	t = Thread(target=process, args=(item,))
	t.start()
	t.join()
Now my requirement is I want to run 5 threads in parallel at a time and if running thread count is less than 5 I want to add another thread to it.
Please help
Reply
#2
It is because you are creating a new list items in the process method and iterate over all of that items. Change the method to:
def process(item):
    print "Sleeping 2 sec from thread %s \n" % item
     
    time.sleep(1)
    #i=len(items)
    print "Finished sleeping from thread %s \n" % item
    try:
        base_path = '/ai2/data/dev/admin/inf/*{}*'
        search_path =  base_path.format(item)
        for f in glob.glob(search_path):
            print ("The directory path is:")
            print f
    except:
        print('error with pattern')
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Error: can't start new thread maha2 0 1,383 Jun-13-2023, 12:26 PM
Last Post: maha2
Question Finding string in list item jesse68 8 1,800 Jun-30-2022, 08:27 AM
Last Post: Gribouillis
  How to seperate dict value? ilknurg 2 1,303 Mar-11-2022, 10:48 AM
Last Post: perfringo
  how to easily create a list of already existing item CompleteNewb 15 3,382 Jan-06-2022, 12:48 AM
Last Post: CompleteNewb
  Remove an item from a list contained in another item in python CompleteNewb 19 5,551 Nov-11-2021, 06:43 AM
Last Post: Gribouillis
  count item in list korenron 8 3,373 Aug-18-2021, 06:40 AM
Last Post: naughtyCat
  Two separate dataframes, two seperate programs stylingpat 2 1,951 Apr-28-2021, 07:56 PM
Last Post: stylingpat
  Time.sleep: stop appending item to the list if time is early quest 0 1,846 Apr-13-2021, 11:44 AM
Last Post: quest
  Seperate output buffer matt_the_hall 2 2,314 Mar-15-2021, 08:44 PM
Last Post: matt_the_hall
  How to run a pytest test for each item in a list arielma 0 2,325 Jan-06-2021, 10:40 PM
Last Post: arielma

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020