Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Write object to file
#1
Hello,

I have developed my code that has several hierarchies of objects, nested in lists, nested in other objects etc, also uses external libraries and is quite complicated to post here.

All I want to do is to create an object and reuse that same object later because it takes quite a lot of time to compute.

I read that I can do a pickle.dump but I get the following error

TypeError: can't pickle _cffi_backend.CData objects
Is there any other way to save my object to a file?
Tried shelve but I'm getting the exact same error as well.
Reply
#2
Please clarify what you want to save - data or how you get the data?
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#3
I'm not sure what you want me to clarify..

I want to save a list of objects.
Each of these object has three attributes, one of them is an integer and the other two are objects. The se lower level objects in turn use external libraries etc..
Reply
#4
Everything in Python is an object.

You wrote that “it takes quite a lot of time to compute”. Quite obvious question is whether you want to save the result of this time consuming computation. But cffi is not my cup of tea.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#5
Yes the time consuming computation is the reason I want to save the whole thing to disk instead of doing this work over again.
Reply
#6
Looking here, https://docs.python.org/3/library/pickle...-unpickled, it seems only native python objects can be pickled, and not all of them. Thus someone wrote an extension to pickle called dill, but it too only pickles native python objects. The message about cffi backend indicates that some of what you are trying to pickle is in C i.e. a C object not fully exposed to python. My newb guess is that you must examine which parts of your object are not pure python and convert them to pure python objects that can be pickled. You will need to code to convert to pure python object and convert from pure python back to your object type.
Reply
#7
Right that was my thought as well, but such examination and conversion is very hard if not impossible in my case since I use a number of external libraries, some of them in turn depending on other libraries.
So I was hoping there's a way to dump the object from memory to disk even if this object is non-native..
Reply
#8
Object serialization is beyond my ken other than what I read on the hitchhiker's guide to python. Offhand I see two problems with the memory idea. First, there is no way that the memory holding your object is contiguous. Secondly, it might not be portable, which may not matter if only you are using it on the same computer. If the authors of the modules you are using provided no methods to save their objects, I think you are out of luck.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How do I write a single 8-bit byte to a file? MysticLord 2 298 Sep-03-2020, 12:27 PM
Last Post: MysticLord
  unable to write to log file Mekala 2 321 Aug-12-2020, 11:02 PM
Last Post: Mekala
  List Won't Write in Text File IILawrenceII 4 399 Jul-17-2020, 11:16 PM
Last Post: j.crater
  Process Data from one csv file and write to another CSV file specific column ajin9581 1 371 Jun-17-2020, 06:00 PM
Last Post: buran
  Write SQLite query result to file hjk6734 1 351 May-27-2020, 12:17 PM
Last Post: menator01
  Append JSON's and write to file faqsap 4 562 May-15-2020, 04:20 PM
Last Post: faqsap
  Print/write to file function tpolim008 4 548 Apr-01-2020, 07:59 PM
Last Post: tpolim008
  Replace XML tag and write to the same file krish216 4 617 Mar-26-2020, 07:39 AM
Last Post: snippsat
  can we write command output to new csv file using Panda package? PythonBeginner_2020 3 591 Mar-13-2020, 12:38 PM
Last Post: ndc85430
  What mode should i open the file to write into a *.msh file Dhanya 0 504 Jan-09-2020, 08:27 AM
Last Post: Dhanya

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020