Python Forum
Reading complex JSON dictionaries
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Reading complex JSON dictionaries
#1
Right, I am writing here to know, if anyone knows a better and more convenient way of using the data, which you get from the response of some API, which gives you a complex Dictionary structure.

Let say I have a class CallAPI, with method get_results(). This will give me the JSON.load results and store it in variable data.

Now let say I call it like this:

api_get_results = CallAPI()
data = api_get_results.get_results().
#Now the data has the complex Dictonarie inside of it for tessting lets imagine this dict
[Player:{
Stats:{ displayeValue #The stats and other keys below "Player" are the ones I want to reach and from those keys, I want to access the dict displayValue.
}
Kills:{ displayValue
}
}
]

The problem I have is if I want to get one specific thing I can do:

get_stats = data['player']['stats']['displayValue']
However, If I have 20 things which I need to access there will be a lot of same code repeating and I wanted to know if anyone knows a better way of pulling out those dict and storing them in a convenient way. Maybe a for cycle? And store those dict into a List?

Just to show by what I mean by repeating, something from my code:

        score = data['stats']['p2']['score']['displayValue']
        top1 = data['stats']['p2']['top1']['displayValue']
        top3 = data['stats']['p2']['top3']['displayValue']
        top5 = data['stats']['p2']['top5']['displayValue']
        top6 = data['stats']['p2']['top6']['displayValue']
        top10 = data['stats']['p2']['top10']['displayValue']
        top12 = data['stats']['p2']['top12']['displayValue']
        top25 = data['stats']['p2']['top25']['displayValue']
        kdRatio = data['stats']['p2']['kd']['displayValue']
Reply
#2
Use intermediate variable and itemgetter for repeating keys:
from operator import itemgetter
p2_data = data['stats']['p2']
value_getter = itemgetter(p2_data['score'])
score = itemgetter(p2_data]['displayValue']
top1 = value_getter(p2_data['top1'])
....
I would also recommend re-organizing your top<N> values into either list or - better, all data into dictionary
json_data = {key: itemgetter(value) for key, value in p2_data.items()}
Test everything in a Python shell (iPython, Azure Notebook, etc.)
  • Someone gave you an advice you liked? Test it - maybe the advice was actually bad.
  • Someone gave you an advice you think is bad? Test it before arguing - maybe it was good.
  • You posted a claim that something you did not test works? Be prepared to eat your hat.
Reply
#3
Will try this out later on in the evening, will need to read more about itemgetters, thanks for the reply!
Reply
#4
I have tried following your suggestion, however, it did not work. Just for more information

import json
import requests
from operator import itemgetter



class API():
    def __init__(self, platform = 'pc'):
        self.api_key = 'myAPIkey'
        self.platform = platform
        self.fortnite_url = 'https://api.fortnitetracker.com/v1/profile/pc/'
        self.headers = {
            'content-type': "application/json",
            'TRN-API-KEY': self.api_key,
          }
    def playerGet(self, player_handle):
        response = requests.request("GET",'https://api.fortnitetracker.com/v1/profile/pc/' + str(player_handle), headers=self.headers)
        data = json.loads(response.text)
        return data

    def playerStats(self,player_handle):
        data = self.playerGet(player_handle)
        
call_api = API()
data = call_api.playerGet("GromiSs")

p2_data = data["stats"]["p2"]

p2_data = {key: value.items() for key, value in p2_data.items()}

print(p2_data)

value_getter = itemgetter(p2_data['score']) # When I call itemgetter the value it returns is a memory address same if I would put it in the sorting for loop above, the values will be changed to a memory address.

print('\n' + str(value_getter))
Now technically I could write this code. # Just as a reminder Stats" is a dictionary, p2 is a dictionary in a dictionary, and the 3 layers are dictionaries too(Score,top1 etc) and those dictionaries have a lot of values inside of it, but the only one that I need is 'displayValue': value.

score = data['stats']['p2']['score']['displayValue']
top1 = data['stats']['p2']['top1']['displayValue']
top3 = data['stats']['p2']['top3']['displayValue']
top5 = data['stats']['p2']['top5']['displayValue']
top6 = data['stats']['p2']['top6']['displayValue']
top10 = data['stats']['p2']['top10']['displayValue']
top12 = data['stats']['p2']['top12']['displayValue']
top25 = data['stats']['p2']['top25']['displayValue']
kdRatio = data['stats']['p2']['kd']['displayValue']
However, I do not want to repeat the code, in addition, I think it would be best to store the values in one variable and if I need to get the specific one, just loop through it.
Example what I mean:
#value here is basically the value which the key 'displayValue' returns
listA = [
{score: 'value here'}
{top1: 'value here'}
{top5: 'value here'} 
...etc
]
and then when I want to use it I can do listA[0] and it will return me the "value" of Score. I hope you understand what I mean.

I just want to reduce the count of variables I need to use, save it faster than just assigning everything to a new variable, save more memory and make the code look "cleaner".
Reply
#5
(Jul-08-2018, 09:31 PM)Gromis Wrote: I have tried following your suggestion, however, it did not work. Just for more information

....
p2_data = {key: value.items() for key, value in p2_data.items()}

print(p2_data)

value_getter = itemgetter(p2_data['score']) # When I call itemgetter the value it returns is a memory ....

OK, I owe you apology for a copy-paste bug. I could have discovered my mistake - if I had a JSON example for testing (for the future - providing data example is a good idea). It should have been

value_getter = itemgetter('displayValue')
p2_data = data['stats']['p2']
p2_data = {key: value_getter(value) for key, value in p2_data.items()}
You probably could have caught my mistake - if you looked into itemgetter

Still, the line after dots in your code was not the one that I suggested.value.items() actually breaks the logic of the code - producing an object of the type dict_values instead of the embedded dict. If you are sure that each embedded dictionary has only one key-value pair, where the key is displayValue, then that would have worked without additional conversions

p2_data = {key: list(value.items())[0] for key, value in p2_data.items()}
You could also write it without itemgetter in that way

p2_data = {key: value['displayValue'] for key, value in data['stats']['p2'].items()}
BTW, requests library provides a shortcut that make importing json redundant - instead of json.loads(response.text) - just response.json()
Test everything in a Python shell (iPython, Azure Notebook, etc.)
  • Someone gave you an advice you liked? Test it - maybe the advice was actually bad.
  • Someone gave you an advice you think is bad? Test it before arguing - maybe it was good.
  • You posted a claim that something you did not test works? Be prepared to eat your hat.
Reply
#6
I appreciate your help Volcano63! I have managed to fix the issue I was having with your reply. Cheers!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  TypeRoor reading json GreenLynx 3 846 May-16-2023, 01:47 PM
Last Post: buran
  Reading Data from JSON tpolim008 2 1,077 Sep-27-2022, 06:34 PM
Last Post: Larz60+
  Initializing, reading and updating a large JSON file medatib531 0 1,770 Mar-10-2022, 07:58 PM
Last Post: medatib531
  Deserialize Complex Json to object using Marshmallow tlopezdh 2 2,115 Dec-09-2021, 06:44 PM
Last Post: tlopezdh
  Help with reading json file hhchenfx 5 4,487 Jul-07-2021, 01:58 PM
Last Post: hhchenfx
  reading json file DrBrownStats 1 1,822 Nov-25-2020, 09:00 AM
Last Post: Larz60+
  Reading a json file ebolisa 2 2,159 Mar-15-2020, 09:24 AM
Last Post: ebolisa
  Reading json behind a login page ebolisa 3 2,422 May-26-2019, 05:15 PM
Last Post: heiner55
  Reading JSON - error jmair 2 2,236 May-22-2019, 07:25 PM
Last Post: jmair
  [split] Reading json from webpage rajesh1997 1 2,197 Feb-05-2019, 12:07 PM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020