Python Forum
Loading multiple JSON files to create a csv
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Loading multiple JSON files to create a csv
#1
Hello,

I need some advice about what I need to do to be able to load multiple files in JSON format to create one big file in CSV.

This is the type of the files which I need to load:

Output:
{ "jobID" : 1235, "requestId" : "aaaa", "deliveryPoints" : [ { "Id" : 456789123, "endpointId" : "99-8x" }, { "Id" : 567891234, "endpointId" : "30-82" }, { "Id" : 678912345, "endpointId" : "30-EB" } ], "numberOfDA" : 4169, "numberOfDC" : 3, "jbStartDate" : 1590943497648, "startDate" : 1590662198157, "eDate" : 1591035600679, "stopDate" : 1590971406468, "State" : "ABORTED", "completedD" : 0, "loadSS" : 0, "loadSR" : 0, "Cycle" : 0, "carouselBN" : 0, "imageCS" : 0, "imageCR" : 0, "missingBS" : 0, "abortedByU" : "EAEA", "abortedRe" : "bla bla bla", "abortedDuringState" : "Prod", "hostname" : [ "xx-xx" ], "missingB" : 0, "totalB" : 0, "failedDA" : [ { "UniID": 456789123, "state": "Validation rejected", "stateDetalil": "network down." }, { "UniID": 567891234, "state": "Validation rejected", "stateDetalil": "network down." }, { "UniID": 678912345, "state": "Validation rejected", "stateDetalil": "network down." }] }
I have tried to use this code but like you can see that I can't integrate "failedDA" like I did with "deliveryPoints".

I have all the files locally but I am struggled to load all the files in one go.


import pandas as pd 
import json 
from pandas.io.json import json_normalize

with open('C:/Users/0li5a3a/Desktop/data/test.json') as f: 
d = json.load(f)
result = json_normalize(d, 'deliveryPoints', ['jobID','requestId', 'Id','numberOfDA','numberOfDC','jbStartDate','startDate','eDate','stopDate','State','completedDA','loadSS','loadSR','Cycle','carouselBN','imageCS','imageCR','missingBS','hostname','missingB','totalB','failedDA'])
result.to_csv('C:/Users/0li5a3a/Desktop/data/test.csv')
Many thanks!
0LI5A3A
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Import multiple CSV files into pandas Krayna 0 1,712 May-20-2021, 04:56 PM
Last Post: Krayna
  JSON file Loading issue punna111 4 8,572 Jun-29-2020, 08:07 AM
Last Post: buran
  Create multiple quiver plots 3D array PengEng 0 1,711 Apr-14-2020, 12:57 PM
Last Post: PengEng
  Append Multiple CSV files Nidhesh 2 2,499 Jul-03-2019, 11:55 AM
Last Post: Nidhesh
  How to extract different data groups from multiple CSV files using python Rafiz 3 3,236 Jun-04-2019, 05:20 PM
Last Post: jefsummers
  Merge JSON files prioritizing the updated values from most recent file nebulae 0 2,536 Apr-17-2019, 10:15 AM
Last Post: nebulae
  Grab columns from multiple files, combine into one jon0852 0 2,014 Feb-12-2019, 02:53 AM
Last Post: jon0852
  Concatenate multiple csv files Oscarca 1 3,062 Nov-05-2018, 11:18 AM
Last Post: Larz60+
  read multiple .xlsx files and text files in a directory BNB 11 25,673 Jun-07-2017, 07:42 AM
Last Post: BNB
  Read CSV Files with multiple headers into Python DataFrame UGuntupalli 12 26,763 Jan-26-2017, 03:07 PM
Last Post: UGuntupalli

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020