Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Decoding a serial stream
#1
Hi,
I am trying to read a UART serial stream, and the data comes in as hex (I think). I’m not actually sure.

Below is a short snippet of what I am getting.

b’\t\x9c\x1d\xc8LX\t\xfdXh\t\xbf\x1e\x8a^X{|\xbe\x85’

I’m pretty sure the “” is a delimiter. And I believe the \n means new line, \t means tab. So those are ascii. I am also guessing that anything starting with \x is a hex value. But that’s where I get lost. Hex \x9c is convertible. But what is \xfdXh?

If anyone can help, or know if there is a python command to parse this, thanks in advance.

Andy
Reply
#2
b’\t\x9c\x1d\xc8LX\t\xfdXh\t\xbf\x1e\x8a^X{|\xbe\x85’ is a bytearray. When you print a bytearray it displays ascii characters when possible, or it uses hexadecimal notation. (\x). Since you are seeing \x in your byte array, it means that the bytearray data is either numbers or Unicode.

If you want to see the bytes as hex values you can do this:
import binascii

x = b'\t\x9c\x1d\xc8LX\t\xfdXh\t\xbf\x1e\x8a^X{|\xbe\x85'
print(binascii.hexlify(x))
Output:
b'099c1dc84c5809fd586809bf1e8a5e587b7cbe85'
If you want to convert it to a Unicode string use .decode().
Reply
#3
(Mar-20-2021, 03:31 PM)deanhystad Wrote: b’\t\x9c\x1d\xc8LX\t\xfdXh\t\xbf\x1e\x8a^X{|\xbe\x85’ is a bytearray. When you print a bytearray it displays ascii characters when possible, or it uses hexadecimal notation. (\x). Since you are seeing \x in your byte array, it means that the bytearray data is either numbers or Unicode.

If you want to see the bytes as hex values you can do this:
import binascii

x = b'\t\x9c\x1d\xc8LX\t\xfdXh\t\xbf\x1e\x8a^X{|\xbe\x85'
print(binascii.hexlify(x))
Output:
b'099c1dc84c5809fd586809bf1e8a5e587b7cbe85'
If you want to convert it to a Unicode string use .decode().
Reply
#4
Thank you for that explanation.

I have used your code as a test. And I have added the decode. The decoded output simply removes the leading "b" and the single quote marks from either end. It didn't really convert it. Can you please explain how to convert this string into readable characters?

============== Code Start ==========================
import binascii

x = b'\t\x9c\x1d\xc8LX\t\xfdXh\t\xbf\x1e\x8a^X{|\xbe\x85'
print(binascii.hexlify(x))
q=binascii.hexlify(x)
print(q.decode())

=========== Code End ===============================

============== Output to Terminal ===============
b'099c1dc84c5809fd586809bf1e8a5e587b7cbe85'
099c1dc84c5809fd586809bf1e8a5e587b7cbe85
Reply
#5
You do not decode the "hexlify", you decode the original bytearray. You also need to specify which encoding was used. The default for .decode is utf-8.

What you really need is a knowledge of what was sent. Is it 20 bytes? Five 32 bit numbers? A Unicode string, and if so, what encoding was used?
Reply
#6
Ha! What is being sent is what I'm trying to figure out. Unfortunately I don't have a clue. It's a solar charger controller with a uart output. I'm trying to decode the stream so I can determine what is being sent.

Any ideas?
Reply
#7
Nope. You need to find a datasheet, a library where someone has already decoded it, or you need to have some mad reconstruction skills. The values may simply be numerical values, but you don't know the size, the range, or the format.

Sometimes you get lucky and the data is a simple ascii or UTF text representation. But most of the time for speed it's a formatted value. Next best thing is you find the chipset and then see if someone has written a driver or decoder module for it already. Even if it's not in python, translating a working driver is much easier than sussing it out blindly.
Reply
#8
It could be something hideous like the way Transducer Data Sheet information (TEDS) is encoded. Those have a header that tells you what template is used to decode the remaining information. Nothing ends on a natural boundary. The template not only specifies format but also scaling and units information.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Decoding lat/long in file name johnmcd 4 270 Mar-22-2024, 11:51 AM
Last Post: johnmcd
  Enigma Decoding Problem krisarmstrong 4 631 Dec-14-2023, 10:42 AM
Last Post: Larz60+
  Microphone stream manipulation Talking2442 1 2,681 Nov-19-2023, 02:08 PM
Last Post: palumanic
  pyserial/serial "has no attribute 'Serial' " gowb0w 9 3,313 Aug-24-2023, 07:56 AM
Last Post: gowb0w
  EEG stream data with mne and brainfolw PaulC 0 461 Aug-22-2023, 03:17 AM
Last Post: PaulC
  json decoding error deneme2 10 3,395 Mar-22-2023, 10:44 PM
Last Post: deanhystad
  flask app decoding problem mesbah 0 2,310 Aug-01-2021, 08:32 PM
Last Post: mesbah
  Best Video Quality And Stream Harshil 2 2,192 Aug-19-2020, 09:03 AM
Last Post: Harshil
  xml decoding failure(bs4) roughstroke 1 2,211 May-09-2020, 04:37 PM
Last Post: snippsat
  python3 decoding problem but python2 OK mesbah 0 1,767 Nov-30-2019, 04:42 PM
Last Post: mesbah

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020