+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Multithreading alternative (/thread-20225.html)
Multithreading alternative - MartinV279 - Aug-01-2019
Inside my project I have a loop of 13 iterations which includes some feature matching. The code needs to run real time, and the only way to achieve this is by doing this loop in parallel. I tried a couple of ways to implement parallel processing with ThreadPool but there was no improvement whatsoever.
from multiprocessing.pool import ThreadPool
import multiprocessing as multip
threads = multip.Pool(multip.cpu_count())
pool = ThreadPool(threads)
def calculateParallel():
results = pool.map(temp_match, list(range(13)))
pool.close()
pool.join()
return resultswhere temp_match is a 30 line image processing procedure which return a ndarray.Is there a way to go around this? From what I read online GIL makes it almost impossible to run multiple CPU threads.
I have seen a lot of posts on forums and tutorials claiming their particular code does the job, but I have not seen any improvement with any of them.
Thanks in advance!
RE: Multithreading alternative - scidam - Aug-01-2019
ThreadPool isn't officially documented. That is, at least, one thing that could be considered suspicious.
You definitely have CPU-bound task here. From here you can find that ThreadPool isn't suitable for your task. Try to use Pool instead. Additionally, you can rewrite/optimize your 35-line image processing procedure using Cython (with no-gil clause), use numba-jit.