Python Forum
UTF-8 decoder reports bad byte that is not there
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
UTF-8 decoder reports bad byte that is not there
#1
i am getting the error:

Output:
Traceback (most recent call last): File "/home/pdh/aptdata.py", line 233, in <module> result=main(argv) File "/home/pdh/aptdata.py", line 206, in main loadinfo(f,data) File "/home/pdh/aptdata.py", line 123, in loadinfo for line in file: File "/usr/lib/python3.5/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
but there is no byte 0x8b anywhere in the whole file. what should i do about this?

the file is one of the files Ubuntu's package management works with. the name of the file is /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_xenial-security_main_binary-amd64_Packages. i have added diagnostic output to the code and appears that the problem is happening just before EOF. the last 2 bytes of the file are 0x0a and 0x0a (an empty line at the end. the is iterating over the lines in the file with for line in file: and every line of the file has been iterated. it's like the utf-8' codec is making this up. maybe some lower level i/o code is leaving garbage in a buffer.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Messages In This Thread
UTF-8 decoder reports bad byte that is not there - by Skaperen - Oct-11-2018, 04:46 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Python best library for Excel reports & review of existing code MasterOfDestr 4 623 Feb-14-2024, 03:39 PM
Last Post: MasterOfDestr
  How can i fix json.decoder.JSONDecodeError ? JgKSuperstar 9 4,600 Oct-30-2021, 11:23 PM
Last Post: JgKSuperstar
  Help with Creating a Script for Automating Reports SunWers 1 1,918 Dec-29-2020, 10:21 PM
Last Post: jjc385
  'utf-8' codec can't decode byte 0xe2 in position 122031: invalid continuation byte tienttt 12 11,518 Sep-18-2020, 10:10 PM
Last Post: tienttt
  coding a decoder... Popkhorn 2 2,106 May-28-2020, 07:26 AM
Last Post: Popkhorn
  Parsing Date/Time from Metar Reports with 6 hourly weather information Lawrence 0 2,332 May-03-2020, 08:15 PM
Last Post: Lawrence
  reset on inactivity (building a morse decoder) gerrit1985 7 3,535 Apr-17-2020, 10:22 AM
Last Post: deanhystad
  'utf-8' codec can't decode byte 0xda in position 184: invalid continuation byte karkas 8 31,652 Feb-08-2020, 06:58 PM
Last Post: karkas
  How do I generate reports in pdf format? okbeat9 2 3,269 Jan-13-2020, 04:38 AM
Last Post: okbeat9
  Python for Reports in HTML or PDF Format ImPyBoy17 1 3,102 Dec-10-2018, 09:35 PM
Last Post: j.crater

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020