Python Forum
sharing memory(read-write) between multiple processes
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
sharing memory(read-write) between multiple processes
#1
I need multiple processes working to be updating(adding data) to an array in memory, so it will be growing large.
the code I'm posting here doesn't do the desired...

import multiprocessing
import ctypes
import numpy as np
shared_array = None


shared_array_base = multiprocessing.Array(ctypes.c_int64, 1 * 10)
shared_array = np.ctypeslib.as_array(shared_array_base.get_obj())
shared_array = shared_array.reshape(1, 10)
# Parallel processing
def my_func(i):
    '''
    anytime a process runs this code, the desire is for the shared array to be stacked with the contents of arr,
    so by running it for example ten times, after all processes return, printing  shape(shared_array)==(11,10)
    //any indication in the right direction how to do that?

    '''
    global shared_array
    arr=np.array([44,44,44,44,44,44,44,44,44,44])
    shared_array=np.vstack((shared_array,arr))
    #print (shared_array)


if __name__ == '__main__':

    pool = multiprocessing.Pool(processes=2, )
    pool.map(my_func, range(10))

    print(shared_array)  # gives [[0 0 0 0 0 0 0 0 0 0]]
Reply
#2
Quote:the code I'm posting here doesn't do the desired...
What in the heck does that mean?

To pass data to/from/between a Process you use a Manager object.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Best way to secure API key when sharing quarinteen 2 289 Jan-19-2024, 04:46 PM
Last Post: deanhystad
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,304 Nov-09-2023, 10:56 AM
Last Post: mg24
Question Special Characters read-write Prisonfeed 1 579 Sep-17-2023, 08:26 PM
Last Post: Gribouillis
  How do I read and write a binary file in Python? blackears 6 6,003 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Read text file, modify it then write back Pavel_47 5 1,499 Feb-18-2023, 02:49 PM
Last Post: deanhystad
  how to read txt file, and write into excel with multiply sheet jacklee26 14 9,504 Jan-21-2023, 06:57 AM
Last Post: jacklee26
  processes shall be parallel flash77 4 1,066 Sep-20-2022, 11:46 AM
Last Post: DeaD_EyE
  Read JSON via API and write to SQL database TecInfo 5 2,108 Aug-09-2022, 04:44 PM
Last Post: TecInfo
  How to write the condition for deleting multiple lines? Lky 3 1,099 Jul-10-2022, 02:28 PM
Last Post: Lky
  Sharing imported modules with Sub Processes? Stubblemonster 2 1,459 May-02-2022, 06:42 AM
Last Post: Stubblemonster

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020