Python Forum
Strange Characters in JSON returned string
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Strange Characters in JSON returned string
#1
Hi,

I'm getting some strange characters at the beginning of an string that I am setting up for a request payload.

In this example I am trying to read a text file containing one data item and then enumerate it.

with open('list.txt', 'r') as f:
             d = {k: v.strip() for (k, v) in enumerate(f, start=1)}
             data = json.dumps(d)
    
             with open('out.txt', 'w') as f_out:  
             f_out.write(data)
the resulting out.txt contains

{"1": "\u00ef\u00bb\u00bf10841911101489"}

the return should read :

{"1": "10841911101489"}

Can anyone help please?

Many thanks,

Fiorano
Reply
#2
hard to say without knowing what's in list.txt, can you provide that?
Reply
#3
It seems as though its due to the encoding of my source txt file.

By default it is output from my source app as a UTF-8 file. If I open and save this as an ANSI text file the process works. Is there anything I can do at the start of my python app to amend the encoding?

Thanks for your help
Reply
#4
You read a file, which has a UTF8 encoding with Byte Order Mark: EF BB BF

Use as encoding utf-8-sig when you read the file.
Then the BOM is stripped away.

The prefix \uxxxx is just a representation for Unicode code points in Json and also for Python.
What you see, are the first three bytes, which defines the Byte Order. Usually this is not used.
I guess you must seek for documents, which are still using the Byte Order Mark.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#5
(Dec-02-2019, 03:56 PM)DeaD_EyE Wrote: You read a file, which has a UTF8 encoding with Byte Order Mark: EF BB BF

Use as encoding utf-8-sig when you read the file.
Then the BOM is stripped away.

The prefix \uxxxx is just a representation for Unicode code points in Json and also for Python.
What you see, are the first three bytes, which defines the Byte Order. Usually this is not used.
I guess you must seek for documents, which are still using the Byte Order Mark.

Perfect Thank You!!!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to access values returned from inquirer cspower 6 699 Dec-26-2023, 09:34 PM
Last Post: cspower
  doing string split with 2 or more split characters Skaperen 22 2,317 Aug-13-2023, 01:57 AM
Last Post: Skaperen
  How do I check if the first X characters of a string are numbers? FirstBornAlbratross 6 1,428 Apr-12-2023, 10:39 AM
Last Post: jefsummers
  [split] Parse Nested JSON String in Python mmm07 4 1,413 Mar-28-2023, 06:07 PM
Last Post: snippsat
  string indices must be integers when parsing Json ilknurg 3 6,205 Mar-10-2022, 11:02 AM
Last Post: DeaD_EyE
  SQLAlchemy Object Missing when Null is returned Personne 1 1,677 Feb-19-2022, 02:50 AM
Last Post: Larz60+
Question [SOLVED] Delete specific characters from string lines EnfantNicolas 4 2,143 Oct-21-2021, 11:28 AM
Last Post: EnfantNicolas
  Getting "name 'get_weather' is not defined error and no json_data returned? trthskr4 6 3,528 Sep-14-2021, 09:55 AM
Last Post: trthskr4
  Libraries installed with pipenv, but ModuleNotFoundError returned jpncsu 2 2,946 Sep-06-2021, 07:24 PM
Last Post: jpncsu
Question convert unlabeled list of tuples to json (string) masterAndreas 4 7,354 Apr-27-2021, 10:35 AM
Last Post: masterAndreas

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020