datasharing between two separated running scripts

[Ispanicuz]

hello everybody,

i need some help understanding what's best solution to share data between two running script.

this is my scenario:

i've a program named sensor.py that when started it creates three separated thread doing somestuff in background and saving result in a string variable json formatted.

this programs start automatically once OS starts.

everything working until here, now it gets worst......

on demand, i'd like to run a simple command line program (std input/output) able to read other program variables running in BG.

i know i can use socket but i dont want to use network, i read about Queue but i don't understand how to let it work with two separates scripts, i also read about named pipe (FIFO) but i understood it works with FileSystem (why i should i/o hdd when i have it in memory?).

thanks for any advice.

[scidam]

You can run key-value store and exchange the data throughout it. E.g.,
install and run Redis server. Redis has python bindings and could be easily accessed from Python. It is quite fast and robust.
Now, one script, that is running in background, could store some data into Redis kv-store,
and the other one could get the data from it.

[Ispanicuz]

You can run key-value store and exchange the data throughout it. E.g.,
install and run Redis server. Redis has python bindings and could be easily accessed from Python. It is quite fast and robust.
Now, one script, that is running in background, could store some data into Redis kv-store,
and the other one could get the data from it.

Redis is often referred as a data structures server. What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a server-client model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.

it uses socket i dont want to use networking(think about a micro controller without a NIC)

[DeaD_EyE]

Then the question comes up, which micro controller and which bus systems/GIPOs the controller has.
The most micro controllers supports a serial connection. The most are based on 3.3V, the Arduino uses 5V.
To connect your micro controller to your PC, you need a signal level converter or you just buy from FTDI an USB2Serial device, which supports 3.3V.

If you want to program with Python on a micro controller, you should look here, for Chinese, here or you google for esp2866 and buy one device for 4$.

I've some developer boards from pycom. The benefit is the supported WiFi connection and the ability to get super small micro frameworks for flask, MQTT and other stuff.

[Ispanicuz]

Then the question comes up, which micro controller and which bus systems/GIPOs the controller has.
The most micro controllers supports a serial connection. The most are based on 3.3V, the Arduino uses 5V.
To connect your micro controller to your PC, you need a signal level converter or you just buy from FTDI an USB2Serial device, which supports 3.3V.

If you want to program with Python on a micro controller, you should look here, for Chinese, here or you google for esp2866 and buy one device for 4$.

I've some developer boards from pycom. The benefit is the supported WiFi connection and the ability to get super small micro frameworks for flask, MQTT and other stuff.

Actually i'm using a raspberry with few I2C sensors, this data is sent to AWS IoT where i trigger some AWS Lambda doing some stuffs at the end of everything i can see them on a HTML page.

my goal is to ssh into the raspberry and using a second script to read those information before they are send out AWS that's why i don't want to use TCP socket i already have underlay networking for other purpose....


Anyway after some googling i think i found the solution using Unix Domain Sockets, same logic as TCP sockets but using internal communication( even if it looks like name pipe(FIFO))

[Gribouillis]

Using module posix_ipc you could share memory segments between the processes.

[Ispanicuz]

Using module posix_ipc you could share memory segments between the processes.

uhmmm this seems to be very close to what i was looking for.... i go to read it thanks m8!

i hope i can use it between different scripts opposite to multiprocessing queue

[Gribouillis]

i hope i can use it between different scripts opposite to multiprocessing queue

You can even use it between a python script and a completely different program. I used it to communicate between python and D programs.