Parsing JSON

[pyStund]

Hi,

I need to take some JSON values that are floats from an AWS DynamoDB and make them variables to use in my script but I keep on running into problems.
The code that works to retrieve them is:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

import boto3, json from boto3.dynamodb.conditions import Key, Attr   # Declare the empty variables ready x1 = 0.0 y1 = 0.0 x2 = 0.0 y2 = 0.0 x3 = 0.0 y3 = 0.0   TABLE_NAME = "xx_data"   # Dynamo connection dynamodb_client = boto3.client('dynamodb', region_name="xxx") dynamodb = boto3.resource('dynamodb', region_name="xxx") table = dynamodb.Table(TABLE_NAME)   response = table.query(   KeyConditionExpression=Key('xx').eq('xx') ) print(response['Items'])

This outputs:

1 2	xxx@xxxx ~ % /usr/bin/python3 /Volumes/xxxx/PyFiles/DynamoData.py [{'device_data': {'y1': '1.663024479783291', 'x1': '0.6768443826584553', 'y2': '1.9438666581951096', 'x2': '1.39031925550876', 'y3': '2.179626982558035', 'x3': '2.0613910485295945'}, 'xx': 'xx'}]

I've tried to use

1 2	data = json.loads(response) data["x1"] = x1

also trying to print just one value leads to the same which makes me think I need to try a different approach

1	print(data ['device_data'][0]['x1'])

But both give me errors about how JSON needs to be a string.

Please can you help point me to resources that are useful for explaining how to parse the output so I can use them in my downstream code.

[Pedroski55]

json.loads() needs a string.
json.load() needs a .json file.

Must be something wrong with your string!

Some advice I got here I believe:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# get the modules import json from pathlib import Path   """ When you load the .json file with data = json.load(f) you have a dictionary of dictionaries Then you can do all the things Python dictionaries can do. """   mypath = '/home/pedro/temp/clients_data_json.json'   # check if the json you want exists chk_file = Path(mypath) if chk_file.is_file():     print(mypath, 'exists, so open it ... \n\n')     with open(mypath) as f:         data = json.load(f) else:     print("File does not exist, so make data an empty dictionary!\n\n")     data = {}

[pyStund]

Hi @Pedroski55
Thank you for your reply.

JSON I specified as the format when I set up the CLI so it pushes the data out as a string.

CLI setup advisory here says:
"json – The output is formatted as a JSON string"

In the boto3 manual it points to me needing to use this to isolate x1 from my previous JSON string as a printed output

1	['device_data'][0]['x1']

Given that it will be string and your advice I used LOAD without the S:

1	data = json.load(response)

Then tried to print it using that boto3 guidance

1	print(data ['device_data'][0]['x1'])

But it returns an error saying: AttributeError: 'dict' object has no attribute 'read'

Thanks again

[ibreeden]

Hi @pyStund , welcome to the forum. I see you studied the manuals of AWS and boto3 quite well. I am sorry to say I didn't, I know nothing about AWS and boto3. But perhaps I can help by analyzing what you posted. I am not sure at all the output is a json string.

1	print(response['Items'])

This outputs:

Output:
[{'device_data': {'y1': '1.663024479783291', 'x1': '0.6768443826584553', 'y2': '1.9438666581951096', 'x2': '1.39031925550876', 'y3': '2.179626982558035', 'x3': '2.0613910485295945'}, 'xx': 'xx'}] 

From this we can learn response is not a string. It must be of type dict (or a subclass of dict).
So this must result in an error:

1	data = json.loads(response)

Error:
Traceback (most recent call last):
  File "/home/ibreeden/PycharmProjects/Forum/kanweg.py", line 9, in <module>
    data = json.loads(response["Items"])
  File "/usr/lib/python3.9/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not list

You did not post the complete error (please always do this in future) but I quess you got something like this.

It still is possible response['Items'] is a string. You could test this with:

1 2	print(type(response)) print(type(response["Items"]))

Output:
<class 'dict'>
<class 'list'>

My guess is: it is a list. In this case there is no need for json. You can easily access your data directly.

1	print(response["Items"][0]["device_data"]["x1"])

Output:
0.6768443826584553

In the other case, when response['Items'] is a string, then you do need to parse it as json.

1 2 3 4 5

print(type(response)) print(type(response["Items"]))   data = json.loads(response["Items"]) print(data[0]["device_data"]["x1"])

Error:
<class 'dict'>
<class 'str'>
0.6768443826584553

Please let us know if this helps you.

[pyStund]

Hi @ibreeden

Thank you, it worked. Using

1 2	x1 = float(response["Items"][0]["device_data"]["x1"]) y1 = float(response["Items"][0]["device_data"]["y1"])

It assigns them back to floats and I can use them in the downstream code.

Great!!