Hello,
I have developed my code that has several hierarchies of objects, nested in lists, nested in other objects etc, also uses external libraries and is quite complicated to post here.
All I want to do is to create an object and reuse that same object later because it takes quite a lot of time to compute.
I read that I can do a pickle.dump but I get the following error
TypeError: can't pickle _cffi_backend.CData objects
Is there any other way to save my object to a file?
Tried shelve but I'm getting the exact same error as well.
Please clarify what you want to save - data or how you get the data?
I'm not sure what you want me to clarify..
I want to save a list of objects.
Each of these object has three attributes, one of them is an integer and the other two are objects. The se lower level objects in turn use external libraries etc..
Everything in Python is an object.
You wrote that “it takes quite a lot of time to compute”. Quite obvious question is whether you want to save the result of this time consuming computation. But cffi is not my cup of tea.
Yes the time consuming computation is the reason I want to save the whole thing to disk instead of doing this work over again.
Looking here,
https://docs.python.org/3/library/pickle...-unpickled, it seems only native python objects can be pickled, and not all of them. Thus someone wrote an extension to pickle called
dill, but it too only pickles native python objects. The message about cffi backend indicates that some of what you are trying to pickle is in C i.e. a C object not fully exposed to python. My newb guess is that you must examine which parts of your object are not pure python and convert them to pure python objects that can be pickled. You will need to code to convert to pure python object and convert from pure python back to your object type.
Right that was my thought as well, but such examination and conversion is very hard if not impossible in my case since I use a number of external libraries, some of them in turn depending on other libraries.
So I was hoping there's a way to dump the object from memory to disk even if this object is non-native..
Object serialization is beyond my ken other than what I read on the
hitchhiker's guide to python. Offhand I see two problems with the memory idea. First, there is no way that the memory holding your object is contiguous. Secondly, it might not be portable, which may not matter if only you are using it on the same computer. If the authors of the modules you are using provided no methods to save their objects, I think you are out of luck.