Jun-20-2019, 06:55 AM
Hi,
In general, I think your problem is suitable for multiprocessing. You have a list of independent data and want to process each data set - which works fine with multiprocessing.
However, for a runtime of ~10 seconds only, it's questionable if you really get any big gains in runtime on multiprocessing. If you want to run faster, I would go here for PyPy or probably the JIT of Numba. Whereas pypy is the easier why: just run your code with pypy instead the the CPython interpreter.
Regards, noisefloor
Quote: Is it possible to set the priority of my processes higher or is it an internal os mechanism which I can not control?You can, but I think not from inside Python - at least I'm not aware how. The way to go depends on the underlying OS. On Linux, you can set process priorities with
nice.In general, I think your problem is suitable for multiprocessing. You have a list of independent data and want to process each data set - which works fine with multiprocessing.
However, for a runtime of ~10 seconds only, it's questionable if you really get any big gains in runtime on multiprocessing. If you want to run faster, I would go here for PyPy or probably the JIT of Numba. Whereas pypy is the easier why: just run your code with pypy instead the the CPython interpreter.
Regards, noisefloor