Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Strange Characters in JSON returned string
#1
Hi,

I'm getting some strange characters at the beginning of an string that I am setting up for a request payload.

In this example I am trying to read a text file containing one data item and then enumerate it.

with open('list.txt', 'r') as f:
             d = {k: v.strip() for (k, v) in enumerate(f, start=1)}
             data = json.dumps(d)
    
             with open('out.txt', 'w') as f_out:  
             f_out.write(data)
the resulting out.txt contains

{"1": "\u00ef\u00bb\u00bf10841911101489"}

the return should read :

{"1": "10841911101489"}

Can anyone help please?

Many thanks,

Fiorano
Quote
#2
hard to say without knowing what's in list.txt, can you provide that?
Quote
#3
It seems as though its due to the encoding of my source txt file.

By default it is output from my source app as a UTF-8 file. If I open and save this as an ANSI text file the process works. Is there anything I can do at the start of my python app to amend the encoding?

Thanks for your help
Quote
#4
You read a file, which has a UTF8 encoding with Byte Order Mark: EF BB BF

Use as encoding utf-8-sig when you read the file.
Then the BOM is stripped away.

The prefix \uxxxx is just a representation for Unicode code points in Json and also for Python.
What you see, are the first three bytes, which defines the Byte Order. Usually this is not used.
I guess you must seek for documents, which are still using the Byte Order Mark.
fioranosnake likes this post
My code examples are always for Python >=3.6.0
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Quote
#5
(Dec-02-2019, 03:56 PM)DeaD_EyE Wrote: You read a file, which has a UTF8 encoding with Byte Order Mark: EF BB BF

Use as encoding utf-8-sig when you read the file.
Then the BOM is stripped away.

The prefix \uxxxx is just a representation for Unicode code points in Json and also for Python.
What you see, are the first three bytes, which defines the Byte Order. Usually this is not used.
I guess you must seek for documents, which are still using the Byte Order Mark.

Perfect Thank You!!!
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  How to use a returned value? t4keheart 12 243 Jan-16-2020, 06:54 AM
Last Post: perfringo
  Replacing characters in a string with a list cjms981 1 142 Dec-30-2019, 10:50 PM
Last Post: micseydel
  Python Parameter inside Json file treated as String dhiliptcs 0 196 Dec-10-2019, 07:28 PM
Last Post: dhiliptcs
  Split a long string into other strings with no delimiters/characters krewlaz 4 156 Nov-15-2019, 02:48 PM
Last Post: ichabod801
  Not all data returned from sys.argv ecdhyne 2 318 Sep-05-2019, 08:27 PM
Last Post: buran
  How to iterate over some characters in a string and the others will stay as it is. ? sodmzs 9 544 Jun-17-2019, 06:45 PM
Last Post: perfringo
  read json string as dict goron 3 383 May-08-2019, 01:43 PM
Last Post: buran
  strange string Skaperen 3 352 Apr-21-2019, 04:36 AM
Last Post: Gribouillis
  I always get 'None' returned. Confused. What did I miss? jpezz 2 321 Apr-07-2019, 10:06 AM
Last Post: jpezz
  Trouble converting JSON String to Dictionary RBeck22 7 1,250 Mar-28-2019, 12:12 PM
Last Post: RBeck22

Forum Jump:


Users browsing this thread: 1 Guest(s)