Python Forum
UTF-8 decoder reports bad byte that is not there
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
UTF-8 decoder reports bad byte that is not there
#1
i am getting the error:

Output:
Traceback (most recent call last): File "/home/pdh/aptdata.py", line 233, in <module> result=main(argv) File "/home/pdh/aptdata.py", line 206, in main loadinfo(f,data) File "/home/pdh/aptdata.py", line 123, in loadinfo for line in file: File "/usr/lib/python3.5/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
but there is no byte 0x8b anywhere in the whole file. what should i do about this?

the file is one of the files Ubuntu's package management works with. the name of the file is /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_xenial-security_main_binary-amd64_Packages. i have added diagnostic output to the code and appears that the problem is happening just before EOF. the last 2 bytes of the file are 0x0a and 0x0a (an empty line at the end. the is iterating over the lines in the file with for line in file: and every line of the file has been iterated. it's like the utf-8' codec is making this up. maybe some lower level i/o code is leaving garbage in a buffer.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Python best library for Excel reports & review of existing code MasterOfDestr 4 596 Feb-14-2024, 03:39 PM
Last Post: MasterOfDestr
  How can i fix json.decoder.JSONDecodeError ? JgKSuperstar 9 4,595 Oct-30-2021, 11:23 PM
Last Post: JgKSuperstar
  Help with Creating a Script for Automating Reports SunWers 1 1,911 Dec-29-2020, 10:21 PM
Last Post: jjc385
  'utf-8' codec can't decode byte 0xe2 in position 122031: invalid continuation byte tienttt 12 11,480 Sep-18-2020, 10:10 PM
Last Post: tienttt
  coding a decoder... Popkhorn 2 2,099 May-28-2020, 07:26 AM
Last Post: Popkhorn
  Parsing Date/Time from Metar Reports with 6 hourly weather information Lawrence 0 2,322 May-03-2020, 08:15 PM
Last Post: Lawrence
  reset on inactivity (building a morse decoder) gerrit1985 7 3,525 Apr-17-2020, 10:22 AM
Last Post: deanhystad
  'utf-8' codec can't decode byte 0xda in position 184: invalid continuation byte karkas 8 31,624 Feb-08-2020, 06:58 PM
Last Post: karkas
  How do I generate reports in pdf format? okbeat9 2 3,259 Jan-13-2020, 04:38 AM
Last Post: okbeat9
  Python for Reports in HTML or PDF Format ImPyBoy17 1 3,099 Dec-10-2018, 09:35 PM
Last Post: j.crater

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020