Python Forum
Convert python dataframe to nested json
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Convert python dataframe to nested json
#1
I have flat csv data loaded into Data frame and trying to build nested json. I was able to build a nested json with orders as list but not able to add few columns like (geo, attributes) as dict within dict (json) and need some pointers on it.

Sample Data -
clientid requestid geo currency date orderid amount quantity attribute1 attribute2
X1 req1 AMER USD 16427166560 1111 600 1 Product A SKU-a
X1 req1 AMER USD 16427166500 1112 1200 1 Product B SKU-b
X1 req1 AMER CAD 16427167500 1113 1500 1 Product A SKU-a
X2 req2 EMEA EUR 16427163500 1114 1000 1 Product A SKU-a

Json O/P expected format -
Output:
{ "request": { "clientid": "X1", "requestid": "req1", "businessunit": { "geo": "AMER" } "currency": "USD", "orders": [{ "date": 16427166560, "order_id": "1111", "amount": "600", "attributes": { "attribute1": "Product A", "attribute2": "SKU-a" } }, { "date": 16427166500, "order_id": "1112", "amount": "1200", "attributes": { "attribute1": "Product B", "attribute2": "SKU-b" } } ] } }
Steps to reproduce

Python Data Frame
df = pd.DataFrame([['X1', 'req1', 'AMER', 'USD', 16427166560, '1111', 600, '1', 'Product A', 'SKU-a'],
                           ['X1', 'req1', 'AMER', 'USD', 16427166500, '1112', 1200, '1', 'Product B', 'SKU-b'],
                           ['X1', 'req1', 'AMER', 'CAD', 16427167500, '1113', 1500, '1', 'Product A', 'SKU-a'],
                           ['X2', 'req2', 'EMEA', 'EUR', 16427163500, '1114', 1000, '1', 'Product A', 'SKU-a']
                           ],
        columns = ['clientid', 'requestid', 'geo', 'currency', 'date', 'order_id', 'amount', 'quantity', 'attribute1', 'attribute2'])
Initial draft which builds nested json -

j = (df.groupby(['clientid', 'requestid', 'geo', 'currency'])
         .apply(lambda x: x[['date', 'order_id', 'amount']].to_dict('records'))
         .reset_index()
         .rename(columns={0: 'orders'})
         .to_json(orient='records'))
Good to have - If we can get separate json file as O/P for each geo/currency in single shot I would appreciate pointers on it.
Reply
#2
Hello All, Please let know if you have any pointers
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Convert dataframe from str back to datafarme Creepy 1 632 Jul-07-2023, 02:13 PM
Last Post: snippsat
  Python Script to convert Json to CSV file chvsnarayana 8 2,513 Apr-26-2023, 10:31 PM
Last Post: DeaD_EyE
  [split] Parse Nested JSON String in Python mmm07 4 1,526 Mar-28-2023, 06:07 PM
Last Post: snippsat
  Converting a json file to a dataframe with rows and columns eyavuz21 13 4,434 Jan-29-2023, 03:59 PM
Last Post: eyavuz21
  Read nested data from JSON - Getting an error marlonbown 5 1,367 Nov-23-2022, 03:51 PM
Last Post: snippsat
  Convert Json to table format python_student 2 5,511 Sep-28-2022, 12:48 PM
Last Post: python_student
  Nested for loops: Iterating over columns of a DataFrame to plot on subplots dm222 0 1,712 Aug-19-2022, 11:07 AM
Last Post: dm222
  Python Split json into separate json based on node value CzarR 1 5,597 Jul-08-2022, 07:55 PM
Last Post: Larz60+
  Convert nested sample json api data into csv in python shantanu97 3 2,815 May-21-2022, 01:30 PM
Last Post: deanhystad
  Cannot convert the series to <class 'int'> when trying to create new dataframe column Mark17 3 8,521 Jan-20-2022, 05:15 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020