Pyserial not reading serial.readline fast enough while using AccelStepper on Arduino

[MartyTinker]

Hello,
New on this forum. Thank you for taking the time to read this post.

I am writing some Python code in VS Code using Pyserial 3.5. along with the with AccelStepper library on my Arduino Uno R3. I am also using the CNC Shield.

[url=https://pyserial.readthedocs.io/en/lates...ml#classes]Link to PySerial[/url]
[url=https://www.airspayce.com/mikem/arduino/...epper.html]Link to AccelStepper[/url]

I almost posted on the Arduino forum but realized my problem was in Python. I say that because when I use Putty, I have no problems at all.

The problem is when my step_x() function in Python sends "x" through the ser.write(b'x') command, if i don't have the time.sleep(1), the return x_Arduino_acknoledge command returns nothing. But when my While True: command asks me for another letter, if I choose 'e', then the print command returns what I asked for the firt time (with the 'x' command). It's always behind one iteration of my while loop. Iteration 3 will return what I chose in iteration 2.
Again, this does not happen with putty.

I want to avoid the time.sleep for two reasons... 1. It adds a delay that will be used thousands of times. 2. Doesn't feel right programming-wise.

In my arduino code, I have the followings lines that sends data back via serial (actually, Serial.printline actually sends the data):

if(pythonData == 'x'){
digitalWrite(Enable_All_Steppers,0); //0=enable all steppers, 1=disable
stepperX.move(1000); //Set the target position relative to the current position
stepperX.runToPosition(); //Moves the motor (with accel./decel.) to target position and blocks until it is at position.
digitalWrite(Enable_All_Steppers,1); //0=enable all steppers, 1=disable
Serial.println("x move done");
}

Please notice stepperX.runToPosition() is suppoese to be a blocking function. It seems to be since the rest of the code seems to be executed after my motor has stopped moving.


Before we get to the Python code, here is something else interesting. Originally, I did not have all the print statements in the function step_x. I added those to debug. Heres's what happens. when I run the script normally, i get in the terminal:

Choose an axis (x: stepper, e:enable, d:disable) : x
Number of bytes in input buffer x1: 0
Number of bytes in input buffer x2: 0
Number of bytes in input buffer x3: 0
Number of bytes in input buffer x4: 0
x move done


When I run in debug mode and slowly run line by line with breakpoints, I get the following:

Choose an axis (x: stepper, e:enable, d:disable) : x
Number of bytes in input buffer x1: 0
Number of bytes in input buffer x2: 13
Number of bytes in input buffer x3: 13
Number of bytes in input buffer x4: 0
x move done

import serial
import time

ser = serial.Serial('COM3', baudrate=115200, timeout=1)
time.sleep(3)

def step_x():
    print("Number of bytes in input buffer x1: " + str(ser.in_waiting))
    ser.write(b'x') #send 'x' to Arduino, 'b' is forbyte
    print("Number of bytes in input buffer x2: " + str(ser.in_waiting))
    time.sleep(1) #Necessary to give Arduino time to send data back to Python
    print("Number of bytes in input buffer x3: " + str(ser.in_waiting))
    x_Arduino_acknoledge =  ser.readline().decode('ascii') #Acknowledge stepping is done. readline uses '\n' as terminator
    print("Number of bytes in input buffer x4: " + str(ser.in_waiting))
    return x_Arduino_acknoledge #return acknowledge text from Arduino

def enable():
    ser.write(b'e') #send 'e' to Arduino
    e_Arduino_acknoledge = ser.readline().decode('ascii') #Acknowledge enable is done.
    return e_Arduino_acknoledge #return acknowledge text from Arduino    

def disable():
    print("Number of bytes in input buffer d1: " + str(ser.in_waiting))
    ser.write(b'd') #send 'd' to Arduino
    print("Number of bytes in input buffer d2: " + str(ser.in_waiting))
    d_Arduino_acknoledge = ser.readline().decode('ascii') #Acknowledge enable is done.
    print("Number of bytes in input buffer d3: " + str(ser.in_waiting))
    return d_Arduino_acknoledge #return acknowledge text from Arduino

while True:
    userInput = input('Choose an axis (x: stepper, e:enable, d:disable) : ')

    if userInput == 'x':
        print(step_x())
    elif userInput == 'e':
        print(enable())
    elif userInput == 'd':
        print(disable())
    else:
        print('Command not recognized')

Final bounus question..
Why do I get:
Number of bytes in input buffer x3: 13
when
"x move done"
has 11 caracters (without quotes). Is it start/stop bits? or is it the quotes?

Thank you all again. This was a lon post.
Martin

[deanhystad]

I would expect 0 bytes in the input buffer. The buffer should be empty immediately after a write and after a read.

There should be no sleep(). Use ser.read() or ser.read_until() with a timeout, and it will wait for the data. I don't know if readline() works or not. It is not a pyserial call.

11 + CR + LF = 13

[MartyTinker]

Thanks for the reply.

I removed the

time.sleep(1) #Necessary to give Arduino time to send data back to Python

but I still get the same result. My terminal window is in the attachments.

x_Arduino_acknoledge = ser.read().decode('ascii')
x_Arduino_acknoledge = ser.read_until().decode('ascii')

Not sure how to implement the timeout. The documentation is not clear to me.
Link to serial.read

My code looks like this

def step_x():
    ser.write(b'x') #send 'x' to Arduino, 'b' is forbyte
    x_Arduino_acknoledge =  ser.read_until().decode('ascii') #Acknowledge stepping is done. readline uses '\n' as terminator
    return x_Arduino_acknoledge #return acknowledge text from Arduino

I'm thinking of putting a special terminating character in my Arduino like '~' and use

ser.read_until('~').decode('ascii')

Might work, but doesn't feel like clean code

[MartyTinker]

I think I found something...

Issue found on Github

When I create my instance, If I use "None", My problem is gone.
ser = serial.Serial('COM3', baudrate=115200, timeout=None)

I tried with 5 seconds (so I don't get locked out) and it works too, but if I put a super long stepping in my Arduino, the timeout unblocks and sends the return after that period.

The moral of the story, use Timeout=None if you want the blocking function in the AccelStepper to do its job.

I hope this helps someone.

[deanhystad]

I would not use runToPosition. The blocking causes a lot of problems, and I don't see any benefit. Use run() instead. If you want to know when the move is complete, periodically check is_running().