Python Forum
How to pass a list by value for manipulation within a process?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to pass a list by value for manipulation within a process?
#1
I am attempting to update a list that lives in an object that I have named Registry. This list is named Registry.registry, and it is a numpy.zeros list. A process calls the list, and attempts to write new data to the list at a specific address. In my test case, I just use the very first index of the list 0x00. The problem that is arising is that the process manipulates a copy of the list, rather than the original list itself. I can see this by printing the object's address during a read or write. So, when I attempt to read the list from a separate process, it only reads yet another copy of the list, not the original that is intended. Here is my code:

import multiprocessing
import numpy as np

class Registry(object):
    def __init__(self, registry_size=128, register_size=16):
        # Registry list to write to
        self.registry = np.zeros((registry_size, register_size), dtype=int)

        # Intermediate buffer for use in bit manipulation
        self.buff = np.zeros((1,16))

    def write16_register(self, reg_addr, data):
        # Convert data to 16 digit binary and sign extend
        data = int(data)
        #print("int(data): " + str(data))
    
        # Split data into 8 bit MSB and LSB
        data_MSB = (data & 0xFF00) >> 8
        data_LSB = (data & 0x00FF)
  
    
        # Load MSB
        self.buff = format(data_MSB, '08b')
        for i in range(0,8,1):
             self.registry[reg_addr][i] = self.buff[i]

        # Load LSB
        self.buff = format(data_LSB, '08b')
        for i in range(8,16,1):
             self.registry[reg_addr][i] = self.buff[i-8]

        # Print statement to see and verify what was written
        print("Write: {0}  to  {1}".format(self.registry[reg_addr], self))

    def read_register(self, reg_addr):
        # Print statement to see and verify what is read before returning
        print("Read:  {0} from {1}".format(self.registry[reg_addr], self))

        return self.registry[reg_addr]

if __name__ == '__main__':
    Main_Registry = Registry()
    p1 = multiprocessing.Process(target=Main_Registry.write16_register, args=(0x00, 0xB594,))
    p2 = multiprocessing.Process(target=Main_Registry.read_register, args=(0x00,))

    p1.start()
    p2.start()

    p1.join()
    p2.join()
Output:
Write: [1 0 1 1 0 1 0 1 1 0 0 1 0 1 0 0] to <__main__.Registry object at 0xb5c7de30> Read: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] from <__main__.Registry object at 0xb5c7de70>
As you can see in the output, the __main__.Registry objects are at different addresses. How can I share the same object between processes to ensure that I am reading and writing to the same list?

I've seen some sources talking about using a Pipe or a Queue to accomplish this?
Reply
#2
Your title says "within a process", but your body looks like you're trying to share it between multiple processes.

I think if you want to share an object between multiprocessing calls, you need to set up a proxy-object. The proxy uses the object in only one of the processes. That said, I've never attempted to use that configuration.
Reply
#3
(Jun-22-2020, 07:52 PM)bowlofred Wrote: Your title says "within a process", but your body looks like you're trying to share it between multiple processes.
Yes, you are correct, I am trying to share this across multiple processes.

(Jun-22-2020, 07:52 PM)bowlofred Wrote: I think if you want to share an object between multiprocessing calls, you need to set up a proxy-object. The proxy uses the object in only one of the processes. That said, I've never attempted to use that configuration.

Do you have a suggestion on how I might better accomplish this? Absent other suggestions I will certainly look into the proxy-object. I am building a machine which has mutiple data points (encoder positions, digital sensor inputs, color sensor, thermocouple, etc) and I am trying to have all of the data stored into this one registry to be accessed by front end developers. I want the Registry to be constantly updating in the background.
Reply
#4
not used it, but in 3.8 there is multiprocessing.shared_memory. I think that's what you need. Look at the docs
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#5
I was able to resolve this issue by getting rid of the numpy.zeroes array, and instead using the multiprocessing.Array datatype. In this way, all the separate processes were able to access and modify the same array, rather than a copy.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to pass encrypted pass to pyodbc script tester_V 0 802 Jul-27-2023, 12:40 AM
Last Post: tester_V
  Problem in list manipulation CyKlop 6 2,238 Oct-18-2021, 09:03 AM
Last Post: DeaD_EyE
  How to pass list of values to a API request URL chetansaip99 0 3,474 Sep-28-2021, 07:37 AM
Last Post: chetansaip99
  Pass by object reference when does it behave like pass by value or reference? mczarnek 2 2,513 Sep-07-2020, 08:02 AM
Last Post: perfringo
  Pass by reference vs Pass by value leodavinci1990 1 2,165 Nov-20-2019, 02:05 AM
Last Post: jefsummers
  How to sharing object between multiple process from main process using Pipe Subrata 1 3,617 Sep-03-2019, 09:49 PM
Last Post: woooee
  list manipulation cameronwood611 3 3,531 Oct-03-2017, 02:58 PM
Last Post: ichabod801
  list or dictionary manipulation dtigue 5 84,344 Jul-21-2017, 03:14 PM
Last Post: ichabod801

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020