Jan-22-2020, 02:31 PM
My Json is as below:
{"Parties":[{"PartyRoleCode":"Customer","PartyBECountry":"GB","PartyBECode":"PRIMARK","PartyBEFunction":"HQ"},{"PartyRoleCode":"Seller","PartyAlias":"TESTING"},{"PartyRoleCode":"Factory","PartyAlias":"TESTING"},{"PartyRoleCode":"Consignee","PartyAlias":"TESTING"},{"PartyRoleCode":"Shipper","PartyAlias":"CN AS VDR"},{"PartyRoleCode":"Buyer","PartyAlias":"TESTING"},{"PartyRoleCode":"NotifyParty","PartyAlias":"TESTING"},{"PartyRoleCode":"Consolidator","PartyAlias":"TESTING"},{"PartyRoleCode":"ShipTo","PartyAlias":"TESTING"},{"PartyRoleCode":"ContainerStuffingLocation","PartyAlias":"TESTING"}]}
for Parties key, I want to iterate and write these data in data frame, How should I do it.Finally I want to write this dataframe in parquet file.
I will be using this python(version 3) code in databricks
If yo observe the attributes are not same for all rows
{"Parties":[{"PartyRoleCode":"Customer","PartyBECountry":"GB","PartyBECode":"PRIMARK","PartyBEFunction":"HQ"},{"PartyRoleCode":"Seller","PartyAlias":"TESTING"},{"PartyRoleCode":"Factory","PartyAlias":"TESTING"},{"PartyRoleCode":"Consignee","PartyAlias":"TESTING"},{"PartyRoleCode":"Shipper","PartyAlias":"CN AS VDR"},{"PartyRoleCode":"Buyer","PartyAlias":"TESTING"},{"PartyRoleCode":"NotifyParty","PartyAlias":"TESTING"},{"PartyRoleCode":"Consolidator","PartyAlias":"TESTING"},{"PartyRoleCode":"ShipTo","PartyAlias":"TESTING"},{"PartyRoleCode":"ContainerStuffingLocation","PartyAlias":"TESTING"}]}
for Parties key, I want to iterate and write these data in data frame, How should I do it.Finally I want to write this dataframe in parquet file.
I will be using this python(version 3) code in databricks
If yo observe the attributes are not same for all rows