Python Forum

Full Version: Write a dictionary with arrays as values into JSON format
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi

I'm working with dictionaries where values are (numpy) arrays, and I'm trying to figure out how to export these dictionaries in an JSON format, and how to read it afterward.

I'm not familiar with JSON specifications and that's why I'm facing some issues : does somebody can advice me on docs to read 'and fto fill my lacks) involving dictionnaries and arrays?

Thanks

Paul

In the following code, only an array is tested
## to write Json file
b = np.ndarray.tolist(MyArray)
json.dump(b, codecs.open(PATH + '/export.json', 'w', encoding='utf-8'), separators=(',', ':'), sort_keys=True, indent=4)

## to read it afterward
with open(PATH + '/export.json', "r") as JsonArray:
    json.load(JsonArray)
Error:
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper
Can you show the full traceback you get? I am not able to reproduce your error and in addition there are so much odd going on here...

1. This error doesn't look like produced from the code you post. From the message
Error:
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper
It looks that the actual code that produce this error you have swapped codecs.open(PATH + '/export.json', 'w', encoding='utf-8') and b when calling json.dump()

Actually, it looks the traceback suggest that you are using json.loads() instead of json.load().


2. There are a lot of odd things you do, e.g.
  • np.ndarray.tolist(MyArray) is same as MyArra.tolist()
  • You are dealing with dict where values are ndarray, but b in the example b is just ndarray
  • Why using codecs module to open the file and not use generic open()? And opening the file inside dump is well bad style, better open the file on separate line and pass the file object handler.


That said here is most basic example
import json
import numpy as np

spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)
eggs = {key:value.tolist() for key, value in eggs.items()}
with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4)
 
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f)
data = {key:np.array(value) for key, value in data.items()}
print(data)
output
Output:
{'some_key': array([1, 2, 3])} {'some_key': array([1, 2, 3])}
More sophisticated approach is to subclass JSONEncoder class.

import json
import numpy as np

class MyJSONEncoder(json.JSONEncoder):         
    def default(self, o):
        try:
            return o.tolist() # works with any object that has .tolist() method
        except AttributeError:
            pass
        # Let the base class default method raise the TypeError
        return json.JSONEncoder.default(self, o)


spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)

with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4, cls=MyJSONEncoder)
 
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f, object_pairs_hook=lambda x: dict((k, np.array(v)) for k, v in x))
print(data)
Output:
{'some_key': array([1, 2, 3])} {'some_key': array([1, 2, 3])}
Of course this can be done differently here and there, but you get the idea, hopefully.
Thanks for the help; it works perfectly
For completeness, let's add one more solution

spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)
with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4, default=lambda x: x.tolist())
  
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f)
data = {key:np.array(value) for key, value in data.items()}
print(data)