Python Forum
Write a dictionary with arrays as values into JSON format
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Write a dictionary with arrays as values into JSON format
#1
Hi

I'm working with dictionaries where values are (numpy) arrays, and I'm trying to figure out how to export these dictionaries in an JSON format, and how to read it afterward.

I'm not familiar with JSON specifications and that's why I'm facing some issues : does somebody can advice me on docs to read 'and fto fill my lacks) involving dictionnaries and arrays?

Thanks

Paul

In the following code, only an array is tested
## to write Json file
b = np.ndarray.tolist(MyArray)
json.dump(b, codecs.open(PATH + '/export.json', 'w', encoding='utf-8'), separators=(',', ':'), sort_keys=True, indent=4)

## to read it afterward
with open(PATH + '/export.json', "r") as JsonArray:
    json.load(JsonArray)
Error:
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper
Reply
#2
Can you show the full traceback you get? I am not able to reproduce your error and in addition there are so much odd going on here...

1. This error doesn't look like produced from the code you post. From the message
Error:
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper
It looks that the actual code that produce this error you have swapped codecs.open(PATH + '/export.json', 'w', encoding='utf-8') and b when calling json.dump()

Actually, it looks the traceback suggest that you are using json.loads() instead of json.load().


2. There are a lot of odd things you do, e.g.
  • np.ndarray.tolist(MyArray) is same as MyArra.tolist()
  • You are dealing with dict where values are ndarray, but b in the example b is just ndarray
  • Why using codecs module to open the file and not use generic open()? And opening the file inside dump is well bad style, better open the file on separate line and pass the file object handler.


That said here is most basic example
import json
import numpy as np

spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)
eggs = {key:value.tolist() for key, value in eggs.items()}
with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4)
 
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f)
data = {key:np.array(value) for key, value in data.items()}
print(data)
output
Output:
{'some_key': array([1, 2, 3])} {'some_key': array([1, 2, 3])}
More sophisticated approach is to subclass JSONEncoder class.

import json
import numpy as np

class MyJSONEncoder(json.JSONEncoder):         
    def default(self, o):
        try:
            return o.tolist() # works with any object that has .tolist() method
        except AttributeError:
            pass
        # Let the base class default method raise the TypeError
        return json.JSONEncoder.default(self, o)


spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)

with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4, cls=MyJSONEncoder)
 
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f, object_pairs_hook=lambda x: dict((k, np.array(v)) for k, v in x))
print(data)
Output:
{'some_key': array([1, 2, 3])} {'some_key': array([1, 2, 3])}
Of course this can be done differently here and there, but you get the idea, hopefully.
buran write Oct-20-2021, 10:38 AM:
One more way to do it: https://python-forum.io/thread-35245-pos...#pid148906
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
Thanks for the help; it works perfectly
Reply
#4
For completeness, let's add one more solution

spam = np.array([1, 2, 3])
eggs = {'some_key':spam}
print(eggs)
with open('export.json', 'w') as f:
    json.dump(eggs, f, separators=(',', ':'), sort_keys=True, indent=4, default=lambda x: x.tolist())
  
## to read it afterward
with open('export.json', "r") as f:
    data = json.load(f)
data = {key:np.array(value) for key, value in data.items()}
print(data)
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  comparing floating point arrays to arrays of integers in Numpy amjass12 0 475 Jul-26-2021, 11:58 AM
Last Post: amjass12
  How to compare two json and write to third json differences with pandas and numpy onenessboy 0 1,932 Jul-24-2020, 01:56 PM
Last Post: onenessboy
  Numpy arrays and compatability with Fortran arrays merrittr 0 940 Sep-03-2019, 03:54 AM
Last Post: merrittr
  Merge JSON files prioritizing the updated values from most recent file nebulae 0 1,657 Apr-17-2019, 10:15 AM
Last Post: nebulae
  Trying to import JSON data into Python/Pandas DataFrame then edit then write CSV Rhubear 0 2,942 Jul-23-2018, 09:50 PM
Last Post: Rhubear

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020