Python Forum
Converting Flattened JSON to Dataframe in Python 2.7
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Converting Flattened JSON to Dataframe in Python 2.7
#1
I am trying to read some data using REST API and write that on a DB table. I have written the below code. But unfortunately, I am kind of stuck with the flattened JSON. Can you please help with a way to convert JSON to Data frame.


import requests
import json
import pandas
from pandas.io.json import json_normalize
from flatten_json import flatten

j_username = 'ABCD'
j_password = '12456'
query = '"id = 112233445566"'
print query
r=requests.get('Url' % query, auth= (j_username,j_password))

print r.json()
first_response = r.json()
string_data = json.dumps(r.json())
normalized_r = json_normalize(r.json())
print flatten(r.json())
r_flattened = flatten(r.json())
r_flattened_str = json.dumps(flatten(r.json()))
print type (flatten(r.json())) 
Output is coming as below

Output is as below
{
'data_0_user-35': u'Xyz',
 'data_0_user-34': None,
 'data_0_user-37': u'CC',
 'data_0_user-36': None,
 'data_0_user-31': u'Regular',
 'data_0_user-33': None, 
 'data_0_user-32': None, 
 'data_0_target-rcyc_id': 0101,
 'data_0_to-mail': None,
 'data_0_closing-version': None, 
 'data_0_user-44': None, 
 'data_0_test-reference': None,
 'data_0_request-server': None, 
 'data_0_target-rcyc_type': u'regular type',
 'data_0_project': None,
 'data_0_user-01': u'Application Name',
 'data_0_user-02': None,
 'data_0_user-03': None, .......
 .......
 ......
 ..... }
Expected out put is as below

data_0_user-35   data_0_user-34  data_0_user-37  .........

XYZ                       None            CC

I have tried the below code

print pandas.DataFrame(r_flattened.items()) 
And I am getting the output as below. This is not what expected but I feel a step closer.

     0                    1
0                 data_0_user-35               Xyz
1                 data_0_user-34                 None
2                 data_0_user-37                   CC
3                 data_0_user-36                 None
4                 data_0_user-31             Regular
Reply


Messages In This Thread
Converting Flattened JSON to Dataframe in Python 2.7 - by ManMan - Jul-12-2017, 01:40 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  converting dataframe to int numpy array glennford49 1 3,146 Apr-04-2020, 06:15 AM
Last Post: snippsat
  Help batch converting .json chosen file to MySQL BrandonKastning 2 3,092 Mar-14-2020, 09:19 PM
Last Post: BrandonKastning
  Transform Facebook Graph API insights JSON to pandas dataframe usman 0 2,481 Mar-03-2020, 05:14 AM
Last Post: usman
  JSON to Dataframe DrX 1 2,516 Feb-19-2020, 07:58 PM
Last Post: DrX
  Converting Dataframe in Python from Object to Float marco_ita 11 17,433 Jan-09-2020, 12:33 PM
Last Post: jefsummers
  Handling escape charters while converting data frame to JSON RahulShukla 0 2,207 Nov-11-2019, 11:22 AM
Last Post: RahulShukla
  Converting string the pandas dataframe chrismc 0 2,853 Jan-24-2019, 11:07 AM
Last Post: chrismc
  Can't store pandas converted json dataframe into mongoDB mahmoud899 1 5,179 Dec-12-2018, 07:45 PM
Last Post: nilamo
  Pandas nested json data to dataframe FrankC 1 11,155 Aug-14-2018, 01:37 AM
Last Post: scidam
  Trying to import JSON data into Python/Pandas DataFrame then edit then write CSV Rhubear 0 4,716 Jul-23-2018, 09:50 PM
Last Post: Rhubear

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020