Write a dictionary with arrays as values into JSON format

[paul18fr]

Hi

I'm working with dictionaries where values are (numpy) arrays, and I'm trying to figure out how to export these dictionaries in an JSON format, and how to read it afterward.

I'm not familiar with JSON specifications and that's why I'm facing some issues : does somebody can advice me on docs to read 'and fto fill my lacks) involving dictionnaries and arrays?

Thanks

Paul

In the following code, only an array is tested

## to write Json file
b = np.ndarray.tolist(MyArray)
json.dump(b, codecs.open(PATH + '/export.json', 'w', encoding='utf-8'), separators=(',', ':'), sort_keys=True, indent=4)

## to read it afterward
with open(PATH + '/export.json', "r") as JsonArray:
    json.load(JsonArray)

Error:
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper

[buran]

Can you show the full traceback you get? I am not able to reproduce your error and in addition there are so much odd going on here...

1. This error doesn't look like produced from the code you post. From the message

Error:
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper

It looks that the actual code that produce this error you have swapped codecs.open(PATH + '/export.json', 'w', encoding='utf-8') and b when calling json.dump()

Actually, it looks the traceback suggest that you are using json.loads() instead of json.load().


2. There are a lot of odd things you do, e.g.

• np.ndarray.tolist(MyArray) is same as MyArra.tolist()
  
• You are dealing with dict where values are ndarray, but b in the example b is just ndarray
  
• Why using codecs module to open the file and not use generic open()? And opening the file inside dump is well bad style, better open the file on separate line and pass the file object handler.
  

That said here is most basic example

import json
import numpy as np

spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)
eggs = {key:value.tolist() for key, value in eggs.items()}
with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4)
 
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f)
data = {key:np.array(value) for key, value in data.items()}
print(data)

output

Output:
{'some_key': array([1, 2, 3])}
{'some_key': array([1, 2, 3])}

More sophisticated approach is to subclass JSONEncoder class.

import json
import numpy as np

class MyJSONEncoder(json.JSONEncoder):         
    def default(self, o):
        try:
            return o.tolist() # works with any object that has .tolist() method
        except AttributeError:
            pass
        # Let the base class default method raise the TypeError
        return json.JSONEncoder.default(self, o)


spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)

with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4, cls=MyJSONEncoder)
 
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f, object_pairs_hook=lambda x: dict((k, np.array(v)) for k, v in x))
print(data)

Output:
{'some_key': array([1, 2, 3])}
{'some_key': array([1, 2, 3])}

Of course this can be done differently here and there, but you get the idea, hopefully.

buran write Oct-20-2021, 10:38 AM:
One more way to do it: https://python-forum.io/thread-35245-pos...#pid148906

[paul18fr]

Thanks for the help; it works perfectly

[buran]

For completeness, let's add one more solution

spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)
with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4, default=lambda x: x.tolist())
  
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f)
data = {key:np.array(value) for key, value in data.items()}
print(data)