Question about deserializing some numbers (bug??) - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: Question about deserializing some numbers (bug??) (/thread-30584.html) |
Question about deserializing some numbers (bug??) - hyoxt121 - Oct-27-2020 Hi! In order to deserialize bytes object, we use pickle.loads(): import pickle import numpy as np pickle.loads(np.float64(0.34103)) and the expected result is like below (because np.float64(0.34103) is not bytes objects, appropriate errors are expected) --------------------------------------------------------------------------- UnpicklingError Traceback (most recent call last) <ipython-input-19-5c07606a60f1> in <module> ----> 1 pickle.loads(np.float64(0.34103)) UnpicklingError: invalid load key, '\xc1'. Here we have some questions that some numbers (it is rare) like 0.34104 prints the following result without errors. pickle.loads(np.float64(0.34104)) => True This occurs only when the converted bytes start with b'\x88 (for example 0.04263, 0.08526, 0.11651 ...) np.float64(0.34104).tobytes() => b'\x88.\xa8o\x99\xd3\xd5?' Can anyone answer whether this issue is Python bugs? Any answer will be highly appreciated. RE: Question about converting numpy float to bytes (bug??) - Gribouillis - Oct-27-2020 It seems that you are attempting an invalid operation. pickle.loads() takes for its first argument a bytes sequence that was created by pickle.dumps() . Here you are passing a numpy.float64 instance. One cannot just pass anything to pickle.loads() and expect it to work.
RE: Question about converting numpy float to bytes (bug??) - hyoxt121 - Oct-27-2020 (Oct-27-2020, 07:43 AM)Gribouillis Wrote: It seems that you are attempting an invalid operation. Regardless of invalid operation, can anyone explain why "pickle.loads(np.float64(0.34104))" prints "True"? RE: Question about deserializing some numbers (bug??) - Gribouillis - Oct-27-2020 It turns out that pickle.loads() starts by converting its argument to BytesIO (unless the argument is a str, see in pickle.py), and it turns out that a numpy float can be converted to a BytesIO >>> import numpy as np >>> thing = np.float64(0.34104) >>> import io >>> file = io.BytesIO(thing) >>> file.getvalue() b'\x88.\xa8o\x99\xd3\xd5?'It means that pickle.loads() thinks that it receives a sequence of bytes. This leaves two questions open
|