JSON Dump and JSON Load

[foxholenoob]

I apologize, the insert code snippet button isn't working.

I am having an issue dumping to json and then reading back in. When I read it back in correctly. In the screenshot you can see visual studio indicating the object is slightly different and when I try to do a for loop through my list it sees the value of "maps" as part of the map list.

My best guess is that I am reading the json back in and its not converting it back to the object correctly.

Here is the code I am using to dump the object to JSON:

fxWarListDump = json.dumps(fxWarConquest.fxWarMaps.maps, default=lambda x: x.__dict__, indent=4)
    with open(fxWarMapList, "w") as outfile:
        outfile.write(fxWarListDump)

Here is the code I am using to read the JSON back in:

with open(fxWarMapList, 'r') as j:
        fxWarConquest.fxWarMaps.maps = json.loads(j.read())

[foxholenoob]

Well boy do I feel silly. Its because of this line:

default=lambda x: x.__dict__

This was causing the object to be export to JSON to a dictionary. Once I removed that the load worked correctly.

[deanhystad]

Use Python tags, not code tags

[foxholenoob]

So to expand on this question. My fix worked on my first class but the second class threw an error saying (object not serializable).

So on my second class I went back to this:

warMapStaticDump = json.dumps(warMapStatic, default=lambda x: x.__dict__, indent=4)

And here is the class in question:

class fxWarMapStatic:
    def __init__(self, mapName):
        self.mapName = mapName
        self.regionId = None
        self.eTag = None
        self.scorchedVictoryTowns = None
        self.lastUpdated = None
        self.version = None
        self.mapTextItems = []

Now I can have multiple of these objects. Each one being unique and read in from a separate JSON file.

So to try and import them back in I ran the following:

        with open(warMapStaticDataDump, 'r') as j:
            fxWarConquest.fxWarMapStatics.append(json.loads(j.read()))

This appears to be working its not and I think my ignorance of Python is where I don't understand what its doing.

If you look at the screenshot below its putting ticks around all the variables of the class. Is this because when I use:

default=lambda x: x.__dict__

Its converting my object to dictionaries and when I bring it back in it places everything correctly but as dictionary objects not variables? Once again, I might be butchering all these terms.

I've done a lot of googling and from what I gathered exporting the object out as JSON can be easy but bringing it back in is where things can get complicated and might not be as easy as a single line of code.

[deanhystad]

json.dump() and json.load() are just the tip of the iceberg. Using json requires a lot more than just that.

A JSON file can only contain numbers, strings, datetime, list and dictionary objects. You cannot dump a fxWarMapStatic object to a json file, and you cannot load a json file and have it return a fxWarMapStatic object. You'll have to write code that can convert your object to things json can serialize. Dumping the __dict__ can do this if the dictionary only contains the types mentioned above, but often you need to write some extra code to make a class "serializable".

Deserializing is a lot more work. When you load a json file you usually get a dictionary or a list. The only types in the object returned by json.load() are numbers, strings. datetime, lists and dictionaries. If you want these to be different classes, you need to write some code that takes the object returned from json.load() and create/restore the objects you want.

[foxholenoob]

Thanks. This is what I was looking for. I think we always hope for a silver bullet cause its so simple to pull data out in a single line of code that you would think it could be brought back in the same way with a single line of code.

I was able to rebuild the object correctly below:

            warMapSerializedData = json.loads(j.read())
            _fxWarMapStatic = fxWarMapStatic(warMapSerializedData["mapName"])
            _fxWarMapStatic.regionId = warMapSerializedData["regionId"]
            _fxWarMapStatic.scorchedVictoryTowns = warMapSerializedData["scorchedVictoryTowns"]
            _fxWarMapStatic.lastUpdated = warMapSerializedData["lastUpdated"]
            _fxWarMapStatic.version = warMapSerializedData["version"]

            fxWarMapStaticMapItems = warMapSerializedData["mapTextItems"]
            for _fxWarMapStaticMapItem in fxWarMapStaticMapItems:
                _fxWarMapStatic.mapTextItems.append(fxWarMapStaticMapItem(_fxWarMapStaticMapItem["text"], _fxWarMapStaticMapItem["x"], _fxWarMapStaticMapItem["y"], _fxWarMapStaticMapItem["mapMarkerType"]))

[deanhystad]

Pickling will do what you wish json would do, but pickling creates a binay file, not a text file.

[buran]

I see the discussion has evolved since I first read this thread in the morning.
Although you think you has resolved the problem, I strongly advise you to read

How to make a class JSON serializable

How to convert JSON data into a Python object?

Using pickle may work, but you should be aware there some security risks. Check the Warning at the top of pickle module docs

Also, couple of style issues with your code - don't use dumps and loads, json.dump and json.load will do

    with open(fxWarMapList, "w") as :
        outfile.dump(fxWarConquest.fxWarMaps.maps, outfile, default=lambda x: x.__dict__, indent=4)

with open(fxWarMapList, 'r') as j:
     fxWarConquest.fxWarMaps.maps = json.load(j)

[foxholenoob]

I see the discussion has evolved since I first read this thread in the morning.
Although you think you has resolved the problem, I strongly advise you to read

How to make a class JSON serializable

I came across that thread during my troubleshooting. Ill take another stab at it but it was very overwhelming considering it has 10+ years of information and for every answer it comes with a caveat.


However, looking at the answers in this thread and thinking about what I am doing overall. My plan to serialize and deserialize my class is probably not even required.

buran write Oct-12-2023, 07:51 AM:
Note, it looks I added a link to deserialisation part as well, while you were reading my response