infinite JSON

[DeaD_EyE]

Don't do this. The task is interesting, but this format is the wrong to do this task and parsing is always a pain and parsing text is very slow.

Use a message queue.

There are many. If you use for example ZMQ, which has Python bindings, it's very short.
Here a example from my memory:


# PUBLISHER

from itertools import count
from time import sleep
import zmq


context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind('tcp://127.0.0.1:5555')
for n in count(1):
    socket.send_multipart([b'frame', str(n).encode()])
    sleep(1)

# SUBSCRIBER

from time import sleep
import zmq


context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.subscribe(b'frame')
socket.connect('tcp://127.0.0.1:5555')
while True:
    typ, frame = socket.recv_multipart()
    print(frame.decode())
    sleep(1)

Instead you can pickle/dump Python objects and send them over the message queue.
Publisher > Subscriber is only one pattern.

There are different approaches for all kind of tasks.
The connection is started inside the application. So zmq is not a server or framework, it's a library.

I use for example ZMQ to send two signals (I/Q) from a radar transceiver to another worker process.
Inside this sensor process i have a time critical read-loop.
Reading not fast enough will crash the program (buffer overflow in hardware.)
The receiving worker process distributes the data to the connected websocket clients.
At the same time the receiving worker process gets also data from a second process (flask app).

I don't use json for zmq, because it is for my task too slow and too much data.

This is only one example. There are many different solutions for it.