Sharing memory across threads / scripts

Sharing memory across threads / scripts - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Sharing memory across threads / scripts (/thread-22109.html)

Sharing memory across threads / scripts - Radek - Oct-30-2019

Hi,

I am trying to gather data from independent python scripts (data acquisition devices) into the main program. I came across multiprocessing, but I still cannot piece it together. Here is the general architecture:

Main program:
* execute device1.py
* execute device2.py

Device1.py
* continuously collects data from a GPIB instrument and places it to shared memory/variable device1.voltage

Device1.py
* continuously collects data from a a DAQ card and places it to shared memory/variable device2.temperature

Main program:
* at predefined interval, it retrieves device1.voltage & device2.temperature and stores them in a log file.
* when main is being shut down, it first termites both device1.py and device2.py

This is a part of a flexible data acquisition suite where different devices can be executed depending on needs. They all produce some kind of data which is collected by the main for recording. At this moment this is my plan, but if there is better way I'd like to hear it.

Thank you.

Best,
Radek

RE: Sharing memory across threads / scripts - Radek - Oct-31-2019

A quick update.

I think I figured it out. I have always assumed that the child process (an instrument script) has to run. I was trying to run as a subpocessor and I was failing to share data with its parent. It finally occurred to me that my device can run as a "function" inside a new script. I can than use multiprocess and shared arguments to pass data.

main_multi.py (parent):

from multiprocessing import Process, Value, Pipe
import time
import os

# If there is a code outside of main() and other functions
# it will be executed here.
import device1

def main():
	reading.append(Value('d', 0.0))
	parent_com, child_com = Pipe()
		
	proc = Process(target=device1.instr, args=(reading, child_com))
	proc.start()
		
	for i in range(20):		
		print( 'Instrument1 = {}'.format(reading.value) )
		time.sleep(2)
	
	print ( 'Main is done ... Sending termination signal to device 1 via Pipe()' )
	parent_com.send(False)
	time.sleep(1)
	print ( 'Exit code for device 1: {}'.format(proc.exitcode) )
	
if __name__ == '__main__':
	main()

device1.py (child)

from multiprocessing import Value
import time
import os
from random import gauss

def instr( result, conn ):
	result.value = 10.0
	running = True	
	while running:
        # check if anything coming via Pipe()
        # when message is "False", loop stops
		if conn.poll():
			running = conn.recv()
			print ( "Message is ", running)
		# generate fake instrument reading
		result.value += gauss(0, 1)
		time.sleep(.1)	
	return 0

def main():
	pass

if __name__ == "__main__":
	main()

I have also managed to extend it to more devices. One can loop over bunch of "devices" and excuse all scrips via for loop. Keep in mind that you will need same number of processes and pipes to connect to all your devices.

Keep in mind that Pipe() is bi-directional. Child can report back to parent when its running, etc.

On a side note. I was a bit confused by the main construct ...

if __name__ == "__main__":
	main()

Here it works well. When child is called by its parent, only "instr" function is called; main() is not executed. However, main() can be used to run it as a stand alone device if needed. <sorry, teaching moment for me :-) >

I hope it helps ...

If there is a better solution I would like to head about it ... I am just starting to code in python (about a month on and off) so I am sure these are more like hacks rather than elegant scripts.

Now I just need a graphical interface for all devices and I am in business!

Peace out!
Radek

P.S.
import os

is there for checking PIDs for various processes ... just saying ...