Python Forum
parallel processing on Mac
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
parallel processing on Mac
#1
It seems the mutiprocessing module is not Mac compatible? The documentation states: "It runs on both Unix and Windows." I guess Unix could include Mac. But when I run the below test code, I see only one Python process running in activity monitor.

import multiprocessing as mp
import math
#import numpy as np

def cube(x):
    a=1
    for i in range(1,100000000):
#        a=np.random.rand(1)[0]
        a=math.cos(i)*a
    return x**3, a

pool = mp.Pool(processes=4)
results = [pool.apply(cube, args=(x,)) for x in range(1,7)]
print(results)

This is getting more confusing.

Depending on the values of the two ranges I sometimes see 4 processes running, and sometimes only 1. But even when 4 processes are indeed running, the sum of their cpu load never gets above 100%, which seems to indicate they don't actually run on separate cores.
Reply
#2
No matter what variations I try (don't use numpy or math for example), I might get multiple processes, but their total processor load never exceeds 100% -- thus all running on a single core.

On the other hand, if I use the parallel processing module pp, http://www.parallelpython.com, I can get multiple Python processes with a processor load at 800%. So, it's not an issue with the Python installation.

But multiprocessing is easy to use and standard, so I wish I could make it work for multiple cores.
Reply
#3
Sorry, I found the issue myself. One has to use pool.apply_async rather than pool.apply
Reply
#4
It could just be that you're using a Pool instead of just starting a couple Processes.  A pool might (...hopefully) only actually starts new processes if they're needed.  For smallish tasks, it makes sense to just do everything in a single thread.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020