Python Forum
Using 2D array with Threadpool
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Using 2D array with Threadpool
#1
Hi guys,

I use 'Threadpool' processing quite a lot but only for 1 dimensional lists of data.I am now trying to implement the same structure but passing a 2D array through the Threadpool.map interface to a function. The problem I have is how to access the values of the array within the function for further use.

array = []

def func(array):
    print(array[0][0] + '   ' array[0][1])
    do stuff here.....
    return results

p = ThreadPool(5)                                                       
results = p.map(func, array)                                     
p.join                                                                  
p.terminate()
This is the typical data in 'array'

Output:
34319 -6 34319 -6 130497 0 187004 -6 187368 0 187340 -1 189637 -6 16272 -4 19633 5 184336 -7 189091 -1 184357 -1 189528 -1
In a 1-dimensional case, the 'array' list simply iterates through the list as it passed to the function. This doesn't seem the case for a 2D array.
Have I got the format wrong?

Any ideas please?

thanks
Reply
#2
I figured it out if anybody interested for future reference:

from multiprocessing.dummy import Pool as ThreadPool
import numpy as np

array = []
def func(array):
    a = np.array(array[0])
    b = np.array(array[1])
    print(a,'   ',b)


p = ThreadPool(1)                                                       
results = p.map(func, array)
p.join                                                                  
p.terminate()
'pool.map' only accepts single lists as parameters so you need to use numpy array. This will then be able to access each element of the array as it is passed to the function.

http://python.omics.wiki/multiprocessing..._arguments
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Cleanest Way to Cancel a ThreadPool SvetlanaofVodianova 0 1,600 Dec-10-2019, 06:25 PM
Last Post: SvetlanaofVodianova

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020