Python Forum
g Null Byte using DictReader
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
g Null Byte using DictReader
#1
I have the below code
stream = io.StringIO(csv_file.stream.read().decode('utf-8-sig'), newline=None) // error is here

reader = csv.DictReader(stream)

list_of_entity = []
line_no, prev_len = 1, 0,

for line in reader:
While executing the above code I got the below error. UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 252862: invalid start byte

Later to fix this I tried the below.
stream = io.StringIO(csv_file.stream.read().decode('unicode_escape'), newline=None)

reader = csv.DictReader(stream)

list_of_entity = []
line_no, prev_len = 1, 0,

for line in reader:// error is here
when i change decode as unicode_escape it thrown the error "_csv.Error: line contains NULL byte" at above highlighted comment line.

There is null byte present in csv, I want to ignore or replace it. can anyone help on this.
Reply
#2
Have you tried plain utf-8 ?
Reply
#3
Hi Larz60+
I tried with utf-8 also but still same error.

_csv.Error: line contains NULL byte
Reply
#4
(May-15-2019, 05:08 AM)eshwinsukhdeve Wrote: I want to ignore or replace it
pass errors argument with value 'ignore' or 'replace' to decode()

the docs
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#5
can you tell me in code how to pass it error argument'ignore' or 'replace' here
Reply
#6
stream = io.StringIO(csv_file.stream.read().decode('utf-8-sig', error='ignore'), newline=None)

and you can also try chardet - https://pypi.org/project/chardet/
for help on what the encoding is
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#7
it says,

TypeError: 'error' is an invalid keyword argument for this function
Reply
#8
sorry, it's errors, not error - my bad
stream = io.StringIO(csv_file.stream.read().decode('utf-8-sig', errors='ignore'), newline=None)
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#9
Still getting same error.
csv.Error: line contains NULL byte

the main issue is in below line of code
for line in reader:
reader object contains null bytes and throwing error here
Reply
#10
(May-15-2019, 05:08 AM)eshwinsukhdeve Wrote: stream = io.StringIO(csv_file.stream.read().decode('utf-8-sig'), newline=None) // error is here
(May-15-2019, 05:08 AM)eshwinsukhdeve Wrote: While executing the above code I got the below error. UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 252862: invalid start byte

My suggestion should fix this error produced by this code
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to express null value klatlap 3 816 Mar-25-2023, 10:40 AM
Last Post: klatlap
  value null when update in json file 3lnyn0 6 3,086 Dec-30-2021, 05:52 PM
Last Post: ndc85430
  'utf-8' codec can't decode byte 0xe2 in position 122031: invalid continuation byte tienttt 12 11,356 Sep-18-2020, 10:10 PM
Last Post: tienttt
  Multiple conditions, one is null moralear27 1 2,167 Sep-13-2020, 06:11 AM
Last Post: scidam
  I didnt get the NULL values salwa17 0 1,555 Jul-10-2020, 02:54 PM
Last Post: salwa17
  Find only the rows containing null values Bhavika 2 2,403 Jun-10-2020, 01:25 PM
Last Post: Bhavika
  'utf-8' codec can't decode byte 0xda in position 184: invalid continuation byte karkas 8 31,481 Feb-08-2020, 06:58 PM
Last Post: karkas
  Array/Jarray problem contains null value? Maverick0013 1 3,933 Aug-10-2018, 12:56 PM
Last Post: Windspar
  4 byte hex byte swap from binary file medievil 7 21,941 May-08-2018, 08:16 AM
Last Post: killerrex
  csv.reader vs csv.dictReader mepyyeti 3 20,719 Mar-20-2018, 08:31 AM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020