Sharing objects in multiple processes

Sharing objects in multiple processes - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: General (https://python-forum.io/forum-1.html)
+--- Forum: Code sharing (https://python-forum.io/forum-5.html)
+--- Thread: Sharing objects in multiple processes (/thread-6278.html)

Sharing objects in multiple processes - duje - Nov-13-2017

Here's a python package I made for sharing complex objects between processes.

git: https://github.com/dRoje/pipe-proxy

The idea is you create a proxy for your object and pass it to a process. Then you use the proxy like you have a reference to the original object. Although you can only use method calls, so accessing object variables is done threw setters and getters.

Say we have an object called ‘example’, creating proxy and proxy listener is easy:

from pipeproxy import proxy 
example = Example() 
exampleProxy, exampleProxyListener = proxy.createProxy(example)

Now you send the proxy to another process.

p = Process(target=someMethod, args=(exampleProxy,)) p.start()

Use it in the other process as you would use the original object (example):

def someMethod(exampleProxy):
    ...
    exampleProxy.originalExampleMethod()
    ...

But you do have to listen to it in the main process:

exampleProxyListener.listen()

Read more and find examples here:

http://matkodjipalo.com/index.php/2017/11/12/proxy-solution-python-multiprocessing/