Bottom Page

Thread Rating:
  • 1 Vote(s) - 4 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Junk values
#1
HI,
i'm reading some values using serial from one energy device... in my out data i'm getting some junk values like
[C]*~ '� � @�ظ��15:49:50 ??. 0 II

i want to remove the junk values and need only the original data from device.. how to remove the junk values?
You like this post
Quote
#2
You can remove junk values if you're able to define which part of the data is made of junk
>>> "This string contains junk values".replace("junk", "")
'This string contains  values'
For example 15:49:00 doesn't look like junk in your output. The much better solution is to understand why the device sends these values and if they have a meaning, but that may be unreachable?
You like this post
Quote
#3
cannot define the exact position of the junk values.. then how can i use it?
You like this post
Quote
#4
How do you know these are junk values? It looks like a copyright notice to me, something like [C]foobar@spam.eggs 15:49:00 bacon but it may be encoded in a exotic codec.
You like this post
Quote
#5
These are not 'junk values', these are 'real values' replaced by unicode special

Quote:The replacement character � (often a black diamond with a white question mark or an empty square box) is a symbol found in the Unicode standard at code point U+FFFD in the Specials table. It is used to indicate problems when a system is unable to render a stream of data to a correct symbol. It is usually seen when the data is invalid and does not match any character:

Consider a text file containing the German word "für" in the ISO-8859-1 encoding (0x66 0xFC 0x72). This file is now opened with a text editor that assumes the input is UTF-8. The first and last byte are valid UTF-8 encodings of ASCII, but the middle byte (0xFC) is not a valid byte in UTF-8. Therefore, a text editor could replace this byte with the replacement character symbol to produce a valid string of Unicode code points. The whole string now displays like this: "f�r".
You like this post
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy
Quote
#6
how to remove these Unicode specials in data?
You like this post
Quote
#7
(Jan-14-2019, 11:22 AM)neethuvp Wrote: how to remove these Unicode specials in data?

From wikipedia article: "Since the replacement is the same for all errors this makes it impossible to recover the original character."

Somehow you need to get original data with original characters and decode appropriately.

I don't see lot of data remaining if you remove specials and some other chars (ظ, ? etc)
You like this post
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  [Selenium] Any Tricks To Block Junk Scripts From Loading? digitalmatic7 0 368 Feb-07-2018, 08:50 PM
Last Post: digitalmatic7

Forum Jump:


Users browsing this thread: 1 Guest(s)