Python Forum
How to share a numpy array between 2 processes on Windows?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to share a numpy array between 2 processes on Windows?
#1
I want to make 2 processes that share a numpy array (one of which writes the array and the other reads it).

It works fine when I make 2 processes with 2 functions like this:


from multiprocessing import Process, Semaphore, shared_memory
import numpy as np
import time

def reader(id, a, shm):
    exst_shm = shared_memory.SharedMemory(name=shm)
    b = np.ndarray(a.shape, dtype=a.dtype, buffer=exst_shm.buf)
        
    time.sleep(2)    
    print('FUNCTION VERSION: ', b[0])   
    

def worker(id, a, shm):
    exst_shm = shared_memory.SharedMemory(name=shm)
    b = np.ndarray(a.shape, dtype=a.dtype, buffer=exst_shm.buf) 
    b[0] += 10 


if __name__ == "__main__":
    a = np.array([0])
    shm = shared_memory.SharedMemory(create=True, size=a.nbytes) 
    c = np.ndarray(a.shape, dtype=a.dtype, buffer=shm.buf)
    
    th1 = Process(target=reader, args=(1, a, shm.name))
    th2 = Process(target=worker, args=(2, a, shm.name))
    
    th1.start()
    th2.start()
    th1.join()
    th2.join()
The code above gives me 10, but when using subclasses it gives me 0 (Windows 10, Python 3.8.7 64bit).

Why do they print out different results? Have I made a mistake or is this a bug?

from multiprocessing import Process, Semaphore, shared_memory
import numpy as np
import time

class Reader(Process):
    def __init__(self, id, a, shm):
        Process.__init__(self)
        
        exst_shm = shared_memory.SharedMemory(name=shm)
        b = np.ndarray(a.shape, dtype=a.dtype, buffer=exst_shm.buf) 
        
        time.sleep(2)    
        print('SUBCLASS VERSION: ', b[0])   
       
class Worker(Process):
    def __init__(self, id, a, shm):
        Process.__init__(self)
            
        exst_shm = shared_memory.SharedMemory(name=shm) 
        b = np.ndarray(a.shape, dtype=a.dtype, buffer=exst_shm.buf) 
        b[0] += 10
        
if __name__ == "__main__":
    a = np.array([0])
    shm = shared_memory.SharedMemory(create=True, size=a.nbytes) 
    c = np.ndarray(a.shape, dtype=a.dtype, buffer=shm.buf)
    
    th1 = Reader(1, a, shm.name)
    th2 = Worker(2, a, shm.name)

    th1.start()
    th2.start()
    th1.join()
    th2.join()
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Convert numpy array to image without loading it into RAM. DreamingInsanity 7 5,857 Feb-08-2024, 09:38 AM
Last Post: paul18fr
  IPython errors for numpy array min/max methods muelaner 1 547 Nov-04-2023, 09:22 PM
Last Post: snippsat
  Timestamp of file changes if a share mapped to a server…. tester_V 34 3,878 Jul-04-2023, 05:19 AM
Last Post: tester_V
  Expand the range of a NumPy array? PythonNPC 0 742 Jan-31-2023, 02:41 AM
Last Post: PythonNPC
  Change a numpy array to a dataframe Led_Zeppelin 3 1,100 Jan-26-2023, 09:01 PM
Last Post: deanhystad
  from numpy array to csv - rounding SchroedingersLion 6 2,152 Nov-14-2022, 09:09 PM
Last Post: deanhystad
  processes shall be parallel flash77 4 1,105 Sep-20-2022, 11:46 AM
Last Post: DeaD_EyE
  numpy.array has no attribute head Led_Zeppelin 1 1,224 Jul-13-2022, 12:56 AM
Last Post: Led_Zeppelin
  Seeing al the data in a dataframe or numpy.array Led_Zeppelin 1 1,135 Jul-11-2022, 08:54 PM
Last Post: Larz60+
  go over and search in numpy array faster caro 7 1,732 Jun-20-2022, 04:54 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020