Python Forum

Full Version: Need help choosing threading method
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I'm coding a pyqt5 interface to exchange data with an arduino through serial communication.

Basically, the program sends a command, the arduino makes a measurement (or whatever) and sends back some values. I want to be able to do this either once or sequentially with minimum time wasted between the commands (e.g. 1000 measurements total, each measurement made every 0.x seconds).

I'm adding some threading to avoid the interface from freezing while it waits for data from the arduino. There are quite a few approaches for threading with pyqt, but I'm not sure which is better suited.

The code below shows the 3 main methods I found online. All three send a "Hello World!" byte string to the arduino, which is echoed back to the program
  • Single Worker: both worker object and thread are destroyed after the thread finishes. This seems suited for single shot commands, but I guess it will create a fair bit of overhead for continuous polling.
  • Loop Worker: same principle as above, except the thread constantly loops over serial.readline() and sends back the results to the main thread until the user gets the worker function to quit with worker.running = False. This seems to fulfill the purpose of reusing the same thread for multiple commands, but its looks a bit hacky (maybe?)
  • QRunnable: as far as I understood, multiple QRunnables can be fed to the thread pool, but I am not sure if there is a significant advantage over the other 2 methods (except that qthreadpool only needs to be instanciated once)

All three methods seem to work with this very simplified example, but I'm wondering which (if any) would be best suited, especially for sequential measurements. Any thoughts?

Arduino code:
void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
}

void loop() {
  // put your main code here, to run repeatedly:
  if(Serial.available() > 0){
    Serial.write(Serial.read());
  } 
}
Python code:
import time

import serial
from PyQt5.QtCore import pyqtSignal, QObject, QThread, QRunnable, pyqtSlot, QThreadPool
from PyQt5.QtWidgets import QMainWindow, QWidget, QPushButton, QVBoxLayout, QCheckBox


class SingleWorker(QObject):
    finished = pyqtSignal()
    result = pyqtSignal(object)

    def __init__(self, ser):
        super(SingleWorker, self).__init__()
        self.ser = ser

    def run(self):
        msg = None
        time.sleep(0.1)
        for i in range(20):
            if self.ser.inWaiting():
                msg = b'SW:' + self.ser.readline()
                break
        self.result.emit(msg)
        self.finished.emit()


class LoopWorker(QObject):
    finished = pyqtSignal()
    result = pyqtSignal(object)

    def __init__(self, ser):
        super(LoopWorker, self).__init__()
        self.ser = ser
        self.running = True

    def run(self):
        time.sleep(0.1)
        while self.running:
            if self.ser.inWaiting():
                msg = b"LW:" + self.ser.readline()
                self.result.emit(msg)
        self.finished.emit()


class WorkerSignals(QObject):
    finished = pyqtSignal()
    result = pyqtSignal(object)


class RunnableWorker(QRunnable):
    def __init__(self, ser):
        super(RunnableWorker, self).__init__()

        self.signals = WorkerSignals()
        self.ser = ser

    @pyqtSlot()
    def run(self):
        msg = None
        time.sleep(0.1)
        for i in range(20):
            time.sleep(0.1)
            if self.ser.inWaiting():
                msg = b'RW:' + self.ser.readline()
                break
        self.signals.result.emit(msg)
        self.signals.finished.emit()


class MainWin(QMainWindow):
    def __init__(self):
        super().__init__()
        self.central_widget = QWidget()
        self.vbox = QVBoxLayout(self.central_widget)
        self.btn1 = QPushButton("Send to single thread", self.central_widget)
        self.check = QCheckBox("Connect loop", self)
        self.btn2 = QPushButton("Send to looping thread", self.central_widget)
        self.btn2.setEnabled(False)
        self.btn3 = QPushButton("Send with QRunnable", self.central_widget)
        self.vbox.addWidget(self.btn1)
        self.vbox.addWidget(self.check)
        self.vbox.addWidget(self.btn2)
        self.vbox.addWidget(self.btn3)
        self.setCentralWidget(self.central_widget)

        self.btn1.pressed.connect(self.send_single)
        self.check.toggled.connect(self.start_loop)
        self.btn2.pressed.connect(self.send_msg)
        self.btn3.pressed.connect(self.qrunnable_send)

        self.ser = serial.Serial("COM4", 9600)
        self.threadpool = QThreadPool()

    def send_single(self):
        self.ser.write(b"hello world!\n")

        self.worker = SingleWorker(self.ser)
        self.thread = QThread()
        self.worker.moveToThread(self.thread)

        self.thread.started.connect(self.worker.run)
        self.worker.finished.connect(self.thread.quit)
        self.worker.finished.connect(self.worker.deleteLater)
        self.thread.finished.connect(self.thread.deleteLater)
        self.thread.finished.connect(self.reset_ui)
        self.worker.result.connect(self.print_msg)

        self.btn1.setEnabled(False)
        self.btn2.setEnabled(False)
        self.btn3.setEnabled(False)

        self.thread.start()

    def send_msg(self):
        self.ser.write(b"hello world!\n")

    def start_loop(self, running):
        if running:
            self.l_worker = LoopWorker(self.ser)
            self.l_thread = QThread()
            self.l_worker.moveToThread(self.l_thread)

            self.l_thread.started.connect(self.l_worker.run)
            self.l_worker.finished.connect(self.l_thread.quit)
            self.l_worker.finished.connect(self.l_worker.deleteLater)
            self.l_thread.finished.connect(self.l_thread.deleteLater)
            self.l_worker.result.connect(self.print_msg)
            self.l_thread.finished.connect(self.reset_ui)

            self.l_thread.start()

            self.btn1.setEnabled(False)
            self.btn2.setEnabled(True)
            self.btn3.setEnabled(False)

        else:
            self.l_worker.running = False

    def qrunnable_send(self):
        self.ser.write(b"hello world!\n")
        worker = RunnableWorker(self.ser)
        worker.signals.result.connect(self.print_msg)
        worker.signals.finished.connect(self.reset_ui)

        self.btn1.setEnabled(False)
        self.btn2.setEnabled(False)
        self.btn3.setEnabled(False)

        self.threadpool.start(worker)

    def receive(self):
        time.sleep(1)
        if self.ser.inWaiting():
            print(self.ser.read_all())

    def print_msg(self, i):
        print(i)

    def reset_ui(self):
        self.btn1.setEnabled(True)
        self.btn2.setEnabled(False)
        self.btn3.setEnabled(True)


if __name__ == "__main__":
    from PyQt5.QtWidgets import QApplication

    app = QApplication([])
    win = MainWin()
    win.show()
    app.exit(app.exec_())
A 9000 baud serial connection is not going to be up to passing thousands of measurements a second.

Do you need a thread? It sounds like you send a message and nearly immediately get back a response. Where is the wait? You could hook that up to run off a timer event.

If you want high speed acquisition of many samples I suggest buffering data on the arduino and having the arduino send a message to the GUI when the buffer is full, or full enough. I have two different scenarios that I use frequently:

1. I have a scope panel that displays real-time signal values. In my controller (your arduino) I have a function that runs periodically at a high rate that copies signal data to a large array (the buffer). My scope panel periodically sends a message to upload the data from the buffer, clearing out room for new data. My buffer is large enough, and the scope updates often enough that I don't have to worry about filling up the buffer.

2. I have a data acquisition system. It is very similar to the scope with a function running in the controller, saving signal data in an buffer. Unlike the scope, it is critical that I don't miss a single data point. When the array buffer starts to get full the controller sends a message to the GUI containing all the buffered data and clears out the buffer to receive new data. The GUI periodically checks if a message was received from the controller, reads the message if one is pending, and writes the data to a file.

Are either of those anything like what you want to do?
It seems my reply didn't get posted the first time...

The arduino won't be passing thousands of measurements a second, I estimate around 0.3s to 0.5s between measurements if not longer. The sentence wasn't very clear in the original post, sorry
Also, the response time for the example above is very fast, but in the final application there will be some delay between the command on pc side and the response from the arduino since the sensor will be moved along a motorized guide rail between measurements.

The final application is used for data acquisition and also has a scope to display the measurements in "real time".
As it is coded today, the python script dumps 100s of coordinates to the arduino buffer which sends back measurements.

One of the issues I have is that the sequence works in an open loop, meaning that the whole thing runs blindly once the measurements have started. However, to have a closed loop, I need the python side to process the incoming response as soon as it arrives on the serial port, hence the use of a thread to continuously poll the serial during the communication phases without freezing the GUI.

Reading your suggestions, I'm also realising this could be done with a QTimer with a short delay although I'm still not convinced it is something that should be done in the main UI thread.
It is easy to try. And if it makes the GUI laggy you can always look into trying threads. I have built Qt apps far more demanding than yours, collecting tens of thousands of signal values a second and having multiple oscilloscope like displays running at the same time. And I am not using doing any threading. But I'm not using a slow serial connection.
(Nov-03-2022, 01:20 PM)deanhystad Wrote: [ -> ]It is easy to try. And if it makes the GUI laggy you can always look into trying threads. I have built Qt apps far more demanding than yours, collecting tens of thousands of signal values a second and having multiple oscilloscope like displays running at the same time. And I am not using doing any threading. But I'm not using a slow serial connection.

The way I understood it from reading online, threading is recommended specifically because the communication is slow and asychronous. In any case I'll definitely try your suggestion, it's all part of learning ;)
shame I won't be able to truly test all this in context before I have rebuilt most of the original application which is a complete mess (think unnecessary operation offloaded to the arduino, huge portions of procedural code executed as subprocesses, global variables everywhere to communicate between processes...)
I'm a big believer in not doing work that doesn't need to be done. Don't seek out problems to solve when you already have plenty on your plate.