parallel processing on Mac

[ahost]

It seems the mutiprocessing module is not Mac compatible? The documentation states: "It runs on both Unix and Windows." I guess Unix could include Mac. But when I run the below test code, I see only one Python process running in activity monitor.

1 2 3 4 5 6 7 8 9 10 11 12 13 14

import multiprocessing as mp import math #import numpy as np   def cube(x):     a=1     for i in range(1,100000000): #        a=np.random.rand(1)[0]         a=math.cos(i)*a     return x**3, a   pool = mp.Pool(processes=4) results = [pool.apply(cube, args=(x,)) for x in range(1,7)] print(results)

---

This is getting more confusing.

Depending on the values of the two ranges I sometimes see 4 processes running, and sometimes only 1. But even when 4 processes are indeed running, the sum of their cpu load never gets above 100%, which seems to indicate they don't actually run on separate cores.

[ahost]

No matter what variations I try (don't use numpy or math for example), I might get multiple processes, but their total processor load never exceeds 100% -- thus all running on a single core.

On the other hand, if I use the parallel processing module pp, http://www.parallelpython.com, I can get multiple Python processes with a processor load at 800%. So, it's not an issue with the Python installation.

But multiprocessing is easy to use and standard, so I wish I could make it work for multiple cores.

[ahost]

Sorry, I found the issue myself. One has to use pool.apply_async rather than pool.apply

[nilamo]

It could just be that you're using a Pool instead of just starting a couple Processes.  A pool might (...hopefully) only actually starts new processes if they're needed.  For smallish tasks, it makes sense to just do everything in a single thread.