Pyserial issues with proper loops and binary

Pyserial issues with proper loops and binary - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Pyserial issues with proper loops and binary (/thread-40474.html)

Pages: 1 2

Pyserial issues with proper loops and binary - jttolleson - Aug-03-2023

So i bought my python programmable Lidar Chip and havent recieved data yet because after opening it up in pyserial i have no idea how to test it...

so i built serial tester code.
with what i have deciphered from the users manual........
[attachment=2486][attachment=2485][attachment=2487]
serialtest.py

#!/usr/bin/env python

import serial
import time

serious = serial.Serial('/dev/ttyUSB0',921600,timeout=.1)
print ('serial port name: '+str(serious.name))        
startCMDchar =  b'0b10101010101010101010101010101010'
deviceaddress = b'00000000'
cmdcodestart = b'0b10'
cmdcodestop = b'0b1111'
cmdcodeok = b'0b10010'
cmdstart = b'0b10101010101010101010101010101010000000000b10'
cmdstop = b'0b10101010101010101010101010101010000000000b1111'
cmdok = b'0b10101010101010101010101010101010000000000b10010'
serious.flush()
serious.write(cmdok)
counter = serious.in_waiting
print (counter)
answer = serious.read(counter)
print (answer)
serious.flush()
serious.write(cmdstart)
counter2 = serious.in_waiting
answer2 = serious.read(counter)
print (counter2)
print (answer2)
serious.write(cmdstop)

I must be way off .....

this is what i am receiving...

[attachment=2488]

RE: Pyserial issues with proper loops and binary - deanhystad - Aug-03-2023

I know that b' ' is a "bytes" literal, but this just means the characters are ascii, not unicode. b'0b1111' is not 15 or 0xF, it is a string of ascii characters '0', 'b', '1', '1', '1', '1'. You are not writing the bytes that you think you are.

I found a python example using a serial port to talk to the LD07.

https://github.com/andreas40239/LdRobot_LD07/blob/main/read_ld07.py

RE: Pyserial issues with proper loops and binary - jttolleson - Aug-04-2023

(Aug-03-2023, 09:04 PM)deanhystad Wrote: I know that b' ' is a "bytes" literal, but this just means the characters are ascii, not unicode. b'0b1111' is not 15 or 0xF, it is a string of ascii characters '0', 'b', '1', '1', '1', '1'. You are not writing the bytes that you think you are.

I found a python example using a serial port to talk to the LD07.

https://github.com/andreas40239/LdRobot_LD07/blob/main/read_ld07.py

Super helpful, yup i am at a stumbling block with the serial communicae and maybe overnight i can get a new program that gets at the data...cool example aswell. I will also run it whole.

RE: Pyserial issues with proper loops and binary - jttolleson - Aug-04-2023

so this is what i will be working with for the next bit....

import serial
import time
import binascii
import math

serious = serial.Serial(port='/dev/ttyUSB0', baudrate=921600, timeout=10.0, bytesize=8, parity='N', stopbits=1)
print ('serial port name: '+str(serious.name))
start = [0xAA,0xAA,0xAA,0xAA,0x01,0x02,0x00,0x00,0x00,0x00,0x03]
ld07_distances = []
ld07_confidences = []

print ('StartMeasurement: '+str(start))
serious.flush()
serious.write(start)
while True:
        counter = serious.in_waiting
        print (counter)
        if(counter >13):
            header = serious.read(4)
            print (header)
            ld07_deviceAddress = int.from_bytes(serious.read(1), byteorder='little')
            ld07_cmdCode = int.from_bytes(serious.read(1), byteorder='little')
            ld07_packetOffsetAddress = int.from_bytes(serious.read(2), byteorder='little')
            ld07_dataLength = int.from_bytes(serious.read(2), byteorder='little')
            ld07_timestamp = int.from_bytes(serious.read(4), byteorder='little')
            print ('ld07_deviceAddress,ld07_cmdCode,ld07_packetOffsetAddress,ld07_dataLength,ld07_timestamp')
            print (ld07_deviceAddress,ld07_cmdCode,ld07_packetOffsetAddress,ld07_dataLength,ld07_timestamp)    
            for measurement in range(16):
                tmpMeasurement = int.from_bytes(serious.read(2), byteorder='little')
                tmpConfidence = (tmpMeasurement >> 9) <<1
                tmpDistance = tmpMeasurement & 0x1ff
                ld07_distances.append(tmpDistance)
                ld07_confidences.append(tmpConfidence)
                print (f"{str(measurement)}:\t{bin(tmpMeasurement)}\t d: {str(tmpDistance)} mm\t Conf: {str(tmpConfidence)}")
serious.flush()

right now when in 'in_waiting' i never count past 0 meaning no bytes were transmitted.
[attachment=2490]

RE: Pyserial issues with proper loops and binary - buran - Aug-04-2023

You pressed Ctrl+C

RE: Pyserial issues with proper loops and binary - jttolleson - Aug-04-2023

(Aug-04-2023, 03:19 AM)buran Wrote: You pressed Ctrl+C

yes i did, to stop the program from printing just zeros, i think i need to look at the manual some more. also.

RE: Pyserial issues with proper loops and binary - deanhystad - Aug-04-2023

Take baby steps. Does this print anything or does it just run for 20 seconds with no output?

import serial

serious = serial.Serial(port='/dev/ttyUSB0', baudrate=921600, timeout=1, bytesize=8, parity='N', stopbits=1)
start = [0xAA,0xAA,0xAA,0xAA,0x01,0x02,0x00,0x00,0x00,0x00,0x03]

serious.write(start)
for _ in range(20):
        if msg := serious.read(1):
             print(msg)

You should take a look at the struct package. You could replace thiis:

import serial

serious.write(start)
while True:
        counter = serious.in_waiting
        print (counter)
        if(counter >13):
            header = serious.read(4)
            print (header)
            ld07_deviceAddress = int.from_bytes(serious.read(1), byteorder='little')
            ld07_cmdCode = int.from_bytes(serious.read(1), byteorder='little')
            ld07_packetOffsetAddress = int.from_bytes(serious.read(2), byteorder='little')
            ld07_dataLength = int.from_bytes(serious.read(2), byteorder='little')
            ld07_timestamp = int.from_bytes(serious.read(4), byteorder='little')

With this:

from serial import Serial
from struct import unpack

serious.write(start)
while True:
    msg = serious.read(size=14)
    if len(msg) != 14:
        raise ValueError(f"Timed out reading message.  Got '{msg}'")
    header, device, cmd, offset, length, tstamp = unpack("<4BBBHHL", msg)
    print(header, device, cmd, offset, length, tstamp)
    # Add code here to read message data based on length and offset values

RE: Pyserial issues with proper loops and binary - jttolleson - Aug-09-2023

Hello all,

I have worked some with the suggestions, i havent gotten data yet however i will continue to work today......right now i am unsure if the data packet is at 11 or 14 bytes in ??? well

here is the current combilned code....it comes up with a timeout...

#!/usr/bin/env python3
import serial
import time
import binascii
import math
from serial import Serial
from struct import unpack
#connect serial and set start vars 
serious = serial.Serial(port='/dev/ttyUSB0', baudrate=921600, timeout=10.0, bytesize=8, parity='N', stopbits=1)
print ('serial port name: '+str(serious.name))
start = [0xAA,0xAA,0xAA,0xAA,0x01,0x02,0x00,0x00,0x00,0x00,0x03]
ld07_distances = []
ld07_confidences = []
#start measurement
print ('StartMeasurement: '+str(start))
serious.flush()
serious.write(start)
#read
while True:
    msg = serious.read(size=14)
    if len(msg) != 14:
        raise ValueError(f"Timed out reading message.  Got '{msg}'")
    header, device, cmd, offset, length, tstamp = unpack("<4BBBHHL", msg)
    print (header, device, cmd, offset, length, tstamp)
    for measurement in range(16):
        tmpMeasurement = int.from_bytes(serious.read(2), byteorder='little') #data byte 11-12 
        tmpConfidence = (tmpMeasurement >> 9) <<1
        tmpDistance = tmpMeasurement & 0x1ff
        ld07_distances.append(tmpDistance)
        ld07_confidences.append(tmpConfidence)
        print (f"{str(measurement)}:\t{bin(tmpMeasurement)}\t d: {str(tmpDistance)} mm\t Conf: {str(tmpConfidence)}")
serious.flush()

As far as moving with baby steps the same issue may be happening, on all scales. i have to get the byte numbers right so the order of the array is represented.

BIG THANKS!!! for the responses on this chip and lidar...

RE: Pyserial issues with proper loops and binary - deanhystad - Aug-10-2023

Writing is 11 bytes. The header (10 bytes) plus the checksum. Reading is variable. You need to read the header first, then the data length field in the header tells you how many data bytes there will be.

I went back and looked at this link: https://github.com/andreas40239/LdRobot_LD07/blob/main/read_ld07.py

After reading "LDROBOT_LD07_Development manual_v1.0_en.pdf" I don't think the example code at this link is very good. Sorry about that, but there's not much to pick from.

For example, I think this is bad code. To write a command, all you should have to provide is the device and the command code.

ld07_startMeasurement = [0xAA,0xAA,0xAA,0xAA,0x01,0x02,0x00,0x00,0x00,0x00,0x03]

ser.write(ld07_startMeasurement)

I would write a function that computes the command code and packs the message. Something like this:

def write_command(port, device, command):
    checksum = (device + command) & 15
    msg_bytes = struct.pack("<LBBHHB", 0xAAAAAAAA, device, command, 0, 0, checksum
    port.write(msg_bytes)

I would do something similar for the read. The first 10 bytes of the message are the start characters, device, command code, data offset and data length. The data length will tell you how many more bytes to read to get the data. The data is followed by a checksum.

def read_data(port):
    # Read the header
    header = port.read(size=10)
    if len(header) < 10:
        raise serial.SerialTimeoutException("Timed out reading header")

    # Verify the header is mostly correct
    start_chars, device, command, offset, size = struct.unpack("<LBBHH", header)
    if start_chars != 0xAAAAAAAA:
        raise serial.SerialException("Start characters don't match")
    if size > 800:
        raise serial.SerialException("Data cannot exceed 800 bytes.")
    # You could also check if the command code is something you recognize.

    # Read the data.  Data is followed by the message checksum.
    data = port-read(size=size+1)
    if len(data) < size+1:
        raise serial.SerialTimeoutException("Timed out reading data")

    # Verify the checksum.  Checksum is the sum of the entire message EXCEPT the 4 byte start character. and checksum.
    checksum = (sum(header[4:]) + sum(data[:-1])) & 15
    if checksum != data[-1]:
        raise serial.SeriaException("Checksum is invalid")

    return device, command, data[:-1]

RE: Pyserial issues with proper loops and binary - jttolleson - Aug-14-2023

Well, time to drop another line: i am definitely working here in a minute or two, its just i do not fully understand your example yet.

I put the code together like this...

#####################################################################################################################
#!/usr/bin/env python3
import serial
import time
import binascii
import math
from serial import Serial
from struct import unpack
#connect serial and set start vars 
#####################################################################################################################
port = '/dev/ttyUSB0'
device = serial.Serial(port, baudrate=921600, timeout=10.0, bytesize=8, parity='N', stopbits=1)
print ('serial port name: '+str(device.name))
device.flush()
#####################################################################################################################
command = [0xAA,0xAA,0xAA,0xAA,0x01,0x02,0x00,0x00,0x00,0x00,0x03]
ld07_distances = []
ld07_confidences = []
#####################################################################################################################
def write_command(port, device, command):
    checksum = (device + command) & 15
    msg_bytes = struct.pack("<LBBHHB", 0xAAAAAAAA, device, command, 0, 0, checksum
    port.write(msg_bytes)
def read_data(port):
    # Read the header
    header = port.read(size=10)
    if len(header) < 10:
        raise serial.SerialTimeoutException("Timed out reading header")
 
    # Verify the header is mostly correct
    start_chars, device, command, offset, size = struct.unpack("<LBBHH", header)
    if start_chars != 0xAAAAAAAA:
        raise serial.SerialException("Start characters don't match")
    if size > 800:
        raise serial.SerialException("Data cannot exceed 800 bytes.")
    # You could also check if the command code is something you recognize.
 
    # Read the data.  Data is followed by the message checksum.
    data = port-read(size=size+1)
    if len(data) < size+1:
        raise serial.SerialTimeoutException("Timed out reading data")
 
    # Verify the checksum.  Checksum is the sum of the entire message EXCEPT the 4 byte start character. and checksum.
    checksum = (sum(header[4:]) + sum(data[:-1])) & 15
    if checksum != data[-1]:
        raise serial.SeriaException("Checksum is invalid")
 
    return device, command, data[:-1], data
#####################################################################################################################
write_command(port,device,command)
data = read_data(port) 
print ('Data: '+data)

when i run it says

ubuntu@ubuntu:~/lidar$ sudo python3 lidartest.py
serial port name: /dev/ttyUSB0
Traceback (most recent call last):
  File "/home/ubuntu/lidar/lidartest.py", line 49, in <module>
    write_command(port,device,command)
  File "/home/ubuntu/lidar/lidartest.py", line 20, in write_command
    checksum = (device + command) & 15
                ~~~~~~~^~~~~~~~~
TypeError: unsupported operand type(s) for +: 'Serial' and 'list'

hmm....i will keep this version and continue to trouble shoot