Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Write object to file
#1
Hello,

I have developed my code that has several hierarchies of objects, nested in lists, nested in other objects etc, also uses external libraries and is quite complicated to post here.

All I want to do is to create an object and reuse that same object later because it takes quite a lot of time to compute.

I read that I can do a pickle.dump but I get the following error

TypeError: can't pickle _cffi_backend.CData objects
Is there any other way to save my object to a file?
Tried shelve but I'm getting the exact same error as well.
Reply
#2
Please clarify what you want to save - data or how you get the data?
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#3
I'm not sure what you want me to clarify..

I want to save a list of objects.
Each of these object has three attributes, one of them is an integer and the other two are objects. The se lower level objects in turn use external libraries etc..
Reply
#4
Everything in Python is an object.

You wrote that “it takes quite a lot of time to compute”. Quite obvious question is whether you want to save the result of this time consuming computation. But cffi is not my cup of tea.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#5
Yes the time consuming computation is the reason I want to save the whole thing to disk instead of doing this work over again.
Reply
#6
Looking here, https://docs.python.org/3/library/pickle...-unpickled, it seems only native python objects can be pickled, and not all of them. Thus someone wrote an extension to pickle called dill, but it too only pickles native python objects. The message about cffi backend indicates that some of what you are trying to pickle is in C i.e. a C object not fully exposed to python. My newb guess is that you must examine which parts of your object are not pure python and convert them to pure python objects that can be pickled. You will need to code to convert to pure python object and convert from pure python back to your object type.
Reply
#7
Right that was my thought as well, but such examination and conversion is very hard if not impossible in my case since I use a number of external libraries, some of them in turn depending on other libraries.
So I was hoping there's a way to dump the object from memory to disk even if this object is non-native..
Reply
#8
Object serialization is beyond my ken other than what I read on the hitchhiker's guide to python. Offhand I see two problems with the memory idea. First, there is no way that the memory holding your object is contiguous. Secondly, it might not be portable, which may not matter if only you are using it on the same computer. If the authors of the modules you are using provided no methods to save their objects, I think you are out of luck.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Read and write active Excel file euras 4 359 Jun-29-2021, 11:16 PM
Last Post: Pedroski55
  Read file, reformat and write new file bryanmartin113 1 337 Jun-08-2021, 09:27 PM
Last Post: Larz60+
  write mariadb table rows query to each file? shams 1 573 Feb-02-2021, 04:10 PM
Last Post: buran
  Read/Write binary file deanhystad 3 726 Feb-01-2021, 10:29 AM
Last Post: Larz60+
  Running A Parser In VSCode - And Write The Results Into A Csv-File apollo 5 991 Jan-14-2021, 08:58 PM
Last Post: snippsat
  How do I write a single 8-bit byte to a file? MysticLord 2 869 Sep-03-2020, 12:27 PM
Last Post: MysticLord
  unable to write to log file Mekala 2 707 Aug-12-2020, 11:02 PM
Last Post: Mekala
  List Won't Write in Text File IILawrenceII 4 758 Jul-17-2020, 11:16 PM
Last Post: j.crater
  Process Data from one csv file and write to another CSV file specific column ajin9581 1 742 Jun-17-2020, 06:00 PM
Last Post: buran
  Write SQLite query result to file hjk6734 1 736 May-27-2020, 12:17 PM
Last Post: menator01

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020