Python Forum

Hi! In order to deserialize bytes object, we use pickle.loads():

import pickle
import numpy as np
pickle.loads(np.float64(0.34103))

and the expected result is like below (because np.float64(0.34103) is not bytes objects, appropriate errors are expected)
---------------------------------------------------------------------------
UnpicklingError Traceback (most recent call last)
<ipython-input-19-5c07606a60f1> in <module>
----> 1 pickle.loads(np.float64(0.34103))

UnpicklingError: invalid load key, '\xc1'.


Here we have some questions that some numbers (it is rare) like 0.34104 prints the following result without errors.
pickle.loads(np.float64(0.34104))
=> True

This occurs only when the converted bytes start with b'\x88 (for example 0.04263, 0.08526, 0.11651 ...)
np.float64(0.34104).tobytes()
=> b'\x88.\xa8o\x99\xd3\xd5?'
Can anyone answer whether this issue is Python bugs?

Any answer will be highly appreciated.

It seems that you are attempting an invalid operation. pickle.loads() takes for its first argument a bytes sequence that was created by pickle.dumps(). Here you are passing a numpy.float64 instance. One cannot just pass anything to pickle.loads() and expect it to work.

(Oct-27-2020, 07:43 AM)Gribouillis Wrote: [ -> ]It seems that you are attempting an invalid operation. pickle.loads() takes for its first argument a bytes sequence that was created by pickle.dumps(). Here you are passing a numpy.float64 instance. One cannot just pass anything to pickle.loads() and expect it to work.

Regardless of invalid operation, can anyone explain why "pickle.loads(np.float64(0.34104))" prints "True"?

It turns out that pickle.loads() starts by converting its argument to BytesIO (unless the argument is a str, see in pickle.py), and it turns out that a numpy float can be converted to a BytesIO

>>> import numpy as np
>>> thing = np.float64(0.34104)
>>> import io
>>> file = io.BytesIO(thing)
>>> file.getvalue()
b'\x88.\xa8o\x99\xd3\xd5?'

It means that pickle.loads() thinks that it receives a sequence of bytes. This leaves two questions open

1. Shouldn't pickle.loads() check more thoroughly its arguments?
   
2. Why are numpy floats convertible to BytesIO?