Python Forum
Threading inside multiprocessing
Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Threading inside multiprocessing
#1
#!/usr/bin/env python2.7
import multiprocessing
from threading import Thread

def do_stuff(i,s):
    print ("work from thread {} - list is {}".format(i,s))

def start_my_work(n,val):
    print ("Start work {} for {}".format(n,val))
    t_source = ['My','Name','Is','John','I','Work','In','IT']
    num_threads = 3
    for s in t_source:
        for i in range(num_threads):
            worker = Thread(target=do_stuff, args=(i,s))
            worker.setDaemon(True)
            worker.start()

jobs = []

p_source = ['hi','there','how','are','you','doing','Jay']

res = [p_source[x:x+2] for x in xrange(0, len(p_source), 2)]

for i,l in enumerate(res):
	for n in range(len(res[i])):
		val = res[i][n]
		p = multiprocessing.Process(target=start_my_work, args=(n,val))
		jobs.append(p)
		p.start()
	
	for p in jobs: #join on all processes ...
		p.join()

print "List processing complete."
Hi All,

I am trying to achieve threading inside each process of multiprocessing. I have 2 lists one for multiprocess (process) & another inside each process. My goal here is to go through p_source & for each process picks up all t_source (parallel) so i have used thread inside process. Also i need to restrict the process spawning to "2".

i have attached my code & am getting the expected output, but its without queue logic. Pls help review & is there any alternate way ? or Is this a right way ?

Regards
Kannan
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Concurrent futures threading running at same speed as non-threading billykid999 13 1,798 May-03-2023, 08:22 AM
Last Post: billykid999
  Tutorials on sockets, threading and multi-threading? muzikman 2 2,113 Oct-01-2021, 08:32 PM
Last Post: muzikman

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020