Python Forum

Full Version: Multithreading alternative
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Inside my project I have a loop of 13 iterations which includes some feature matching. The code needs to run real time, and the only way to achieve this is by doing this loop in parallel. I tried a couple of ways to implement parallel processing with ThreadPool but there was no improvement whatsoever.

from multiprocessing.pool import ThreadPool
import multiprocessing as multip

threads = multip.Pool(multip.cpu_count())
pool = ThreadPool(threads)


def calculateParallel():
    results = pool.map(temp_match, list(range(13)))
    pool.close()
    pool.join()
    return results
where temp_match is a 30 line image processing procedure which return a ndarray.
Is there a way to go around this? From what I read online GIL makes it almost impossible to run multiple CPU threads.
I have seen a lot of posts on forums and tutorials claiming their particular code does the job, but I have not seen any improvement with any of them.
Thanks in advance!
ThreadPool isn't officially documented. That is, at least, one thing that could be considered suspicious.
You definitely have CPU-bound task here. From here you can find that ThreadPool isn't suitable for your task. Try to use Pool instead. Additionally, you can rewrite/optimize your 35-line image processing procedure using Cython (with no-gil clause), use numba-jit.