extract specific data from a group of json-files

[DeaD_EyE]

First you should know the data structure of the json file.
You can investigate it, if you open the file with Python and use json.loads() on the open file.

import json

with open('your_data0001.json') as fd:
    data = json.loafs(fd)

Usually json data is a dictionary with subdictionaries and lists.
To show the keys, you can use list(data.keys()).
If you find the right key, you can dig deeper.

For example if you have the key 'metadata', accessing it, is very easy:

data['metadata']

The value of 'metadata' could be a list or a dict or something else (int, float, str).


After you know the structure, you can write a transformer function for it.
It should transform the json data into the form you want to have.

This is just an example and do not have to fit on your data.

def transformer(mapping_from_json):
    """
    A generator which takes a mapping (dict)
    and yields name, age, active
    """
    for items in mapping_from_json['results']['metadata']:
        # if metadata is a list
        for element in items:
            name = element.get('name', 'NO NAME')
            age = element.get('age', 0)
            active = element.get('active', False)
            yield (name, age, active)

I used the generator, because the logic is easier to understand. Always if a yield is in a function, then it's returning a generator, if you call the function. Iterating over the generator, yields the elements.

To consume it, just use something, which takes iterables or use a for loop.

list(transformer(my_dict))