Dec-19-2018, 08:01 AM
I have a sample code which uses joblibs and multiprocessing modules. The code works fine when run from the command line, but when I package it as an executable using Pyinstaller, the multiple processes spawn as new instances infinitely (creating new child process id in backend).
After scoping lot of sources I was suggested to use
This resolved the infinite invocation of my main script, but after completion of the code, there was a left out child process still running in background invoked with a new process id always with below process name :
Code :
But to no use.
After scoping lot of sources I was suggested to use
multiprocessing.freeze_support()
at the beginning of my if __name__ == "__main__":
block.This resolved the infinite invocation of my main script, but after completion of the code, there was a left out child process still running in background invoked with a new process id always with below process name :
Output:user 2741 1 28 14:25 pts/5 00:00:00 /home/user/ParallelProcessing -E -s -B -S -c from multiprocessing.semaphore_tracker import main;main(5)
After few seconds (Process ID changes)Output:user 2745 1 28 14:25 pts/5 00:00:00 /home/user/ParallelProcessing -E -s -B -S -c from multiprocessing.semaphore_tracker import main;main(5)
I would just like to know what additional part am I missing in my code to avoid such issues.Code :
from joblib import Parallel, delayed import multiprocessing import time inputs = range(10) def processInput(i): print("Started : ",i) time.sleep(i) print("Completed : ",i) return i * i def main(): num_cores = multiprocessing.cpu_count() backend = 'threading' print(num_cores) results = Parallel(n_jobs=num_cores, backend=backend)(delayed(processInput)(i) for i in inputs) print(results) if __name__ == "__main__": multiprocessing.freeze_support() main()Have even tried handling via :
backend=threading
argument during Parallel() call. But to no use.