Python Forum
Reading json file as pandas data frame? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: Reading json file as pandas data frame? (/thread-8055.html)



Reading json file as pandas data frame? - Alberto - Feb-04-2018

Dear Python Users,

I am using python 3.6 and trying to download json file (350 MB) as pandas dataframe using the code below. However, I get the following error:

Error:
data_json_str = "[" + ",".join(data) + "] "TypeError: sequence item 0: expected str instance, bytes found
How can I fix the error?

import pandas as pd

# read the entire file into a python array
with open('C:/Users/Alberto/nutrients.json', 'rb') as f:
   data = f.readlines()

# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)

# each element of 'data' is an individual JSON object.
# i want to convert it into an *array* of JSON objects
# which, in and of itself, is one large JSON object
# basically... add square brackets to the beginning
# and end, and have all the individual business JSON objects
# separated by a comma
data_json_str = "[" + ",".join(data) + "]"

# now, load it into pandas
data_df = pd.read_json(data_json_str)



RE: Reading json file as pandas data frame? - snippsat - Feb-05-2018

data_json_str is bytes pd.read_json need string.
>>> data_json_str = b'hello'
>>> type(data_json_str)
<class 'bytes'>

>>> s = data_json_str.decode()
>>> s
'hello'
>>> type(s)
<class 'str'>