JSON vs PYON

[wavic]

There is this module called jsonpickle which, according to the author, can serialize almost any Python object. It's been here for quite some time but I've never needed it

[Skaperen]

is jsonpickle directly compatible with JavaScript? can loading jsonpickle restore all the Python objects it dumps to the same exact thing they were before?

my goal with PYON is serialize to a format that can be inserted in Python source code or use other means of parsing Python source like eval() or exec() ... and ... have improved human readability (one example is that it sorts dictionary keys ... an implementation thing).

[wavic]

I think it can but you can test it. As I remember it uses json to store the objects so it should be JS compatible

[Skaperen]

there are data types Python has that JavaScript does not have. if jsonpickle supports those types then it is not fully compatible with JavaScript. maybe it is putting pickle format in a string and putting that string in JSON? i have no idea if JavaScript supports control characters or the whole 0..127 or 0..255 range of character values. i think PYON is the way to go for Python compatibility unless you also need JavaScript compatibility.

[ndc85430]

Note that JSON really isn't about JavaScript at all. It's a serialisation format, used for exchanging data between systems. Those systems need not be written in JavaScript or any related language because libraries to covert JSON to and from different kinds of data structures (for example dictionaries in Python, or classes in Java) exist for many, many languages. That quite a simple format is widely used in a variety of domains doesn't suggest to me that we need another really.

What kinds of types are you wanting to encode/decode that you think JSON wouldn't be suitable for?

[Skaperen]

types like boolean. decimal.Decimal. complex.

[wavic]

It's not a pickle format. The module is called that way so one can figure its purpose.
It stores all the data about an object as JSON.

Output:
In [1]: import jsonpickle                                                                                                                                                                                                                                                                                                                                                                                                                                                                 In [2]: import decimal                                                                                                                                                                                                                                                                                                                                                                                                                                                                    In [3]: from pprint import pprint                                                                                                                                                                                                                                                                                                                                                                                                                                                         In [4]: class Test:                                                                                                                                                                                                                             ...:     def __init__(self):                                                                                                                                                                                                                 ...:         self.bul  = True                                                                                                                                                                                                                ...:         self.pi = decimal.Decimal('3.14')                                                                                                                                                                                               ...:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   In [5]: obj = Test()                                                                                                                                                                                                                                                                                                                                                                                                                                                                      In [6]: pickled = jsonpickle.encode(obj)                                                                                                                                                                                                                                                                                                                                                                                                                                                  In [7]: pprint(pickled)                                                                                                                                                                                                                      ('{"bul": true, "pi": {"py/reduce": [{"py/type": "decimal.Decimal"}, '                                                                                                                                                                        '{"py/tuple": ["3.14"]}]}, "py/object": "__main__.Test"}')                                                                                                                                                                                                                                                                                                                                                                                                                               In [8]: del obj                                                                                                                                                                                                                                                                                                                                                                                                                                                                           In [9]: obj                                                                                                                                                                                                                                  ---------------------------------------------------------------------------                                                                                                                                                                  NameError                                 Traceback (most recent call last)                                                                                                                                                                  <ipython-input-9-c88944e91e53> in <module>                                                                                                                                                                                                   ----> 1 obj                                                                                                                                                                                                                                                                                                                                                                                                                                                                               NameError: name 'obj' is not defined                                                                                                                                                                                                                                                                                                                                                                                                                                                      In [10]: obj = jsonpickle.decode(pickled)                                                                                                                                                                                                                                                                                                                                                                                                                                                 In [11]: obj                                                                                                                                                                                                                                 Out[11]: <__main__.Test at 0x7ff27f1f9f90>                                                                                                                                                                                                                                                                                                                                                                                                                                                In [12]: type(obj)                                                                                                                                                                                                                           Out[12]: __main__.Test                                                                                                                                                                                                                                                                                                                                                                                                                                                                    In [13]: obj.bul                                                                                                                                                                                                                             Out[13]: True                                                                                                                                                                                                                                                                                                                                                                                                                                                                             In [14]: obj.pi                                                                                                                                                                                                                              Out[14]: Decimal('3.14')

---

I used output tags here because Python tags put all in one line.

[Gribouillis]

In serializing arbitrary objects, one of the main issues is that of cyclic references. I doubt any of these jsonxxx modules can handle this.

[Skaperen]

i have read that many things cannot handle cyclic references. i have not done that in my PYON code. i do intend to add some code to detect such cycles and avoid them, but i have not worked out ways to represent and reconstruct cycles.

that does look like a different way of defining types that is not inherintly JSON strung out inside JSON. so i would not call it JSON. it's not Python, either. it can't be fully utilized in either JavaScript or Python without something to decode it. OTOH, PYON is fully Python (except for cycles), leaving it up to other languages to develop their own tools for it, if it becomes popular enough. my implementation is still not done. i need to add a few things after i redo it all as a generator or coroutine. then dump output would just wrap around the generator.