Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Write object to file
#1
Hello,

I have developed my code that has several hierarchies of objects, nested in lists, nested in other objects etc, also uses external libraries and is quite complicated to post here.

All I want to do is to create an object and reuse that same object later because it takes quite a lot of time to compute.

I read that I can do a pickle.dump but I get the following error

TypeError: can't pickle _cffi_backend.CData objects
Is there any other way to save my object to a file?
Tried shelve but I'm getting the exact same error as well.
Reply
#2
Please clarify what you want to save - data or how you get the data?
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#3
I'm not sure what you want me to clarify..

I want to save a list of objects.
Each of these object has three attributes, one of them is an integer and the other two are objects. The se lower level objects in turn use external libraries etc..
Reply
#4
Everything in Python is an object.

You wrote that “it takes quite a lot of time to compute”. Quite obvious question is whether you want to save the result of this time consuming computation. But cffi is not my cup of tea.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#5
Yes the time consuming computation is the reason I want to save the whole thing to disk instead of doing this work over again.
Reply
#6
Looking here, https://docs.python.org/3/library/pickle...-unpickled, it seems only native python objects can be pickled, and not all of them. Thus someone wrote an extension to pickle called dill, but it too only pickles native python objects. The message about cffi backend indicates that some of what you are trying to pickle is in C i.e. a C object not fully exposed to python. My newb guess is that you must examine which parts of your object are not pure python and convert them to pure python objects that can be pickled. You will need to code to convert to pure python object and convert from pure python back to your object type.
Reply
#7
Right that was my thought as well, but such examination and conversion is very hard if not impossible in my case since I use a number of external libraries, some of them in turn depending on other libraries.
So I was hoping there's a way to dump the object from memory to disk even if this object is non-native..
Reply
#8
Object serialization is beyond my ken other than what I read on the hitchhiker's guide to python. Offhand I see two problems with the memory idea. First, there is no way that the memory holding your object is contiguous. Secondly, it might not be portable, which may not matter if only you are using it on the same computer. If the authors of the modules you are using provided no methods to save their objects, I think you are out of luck.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Last record in file doesn't write to newline gonksoup 3 364 Jan-22-2024, 12:56 PM
Last Post: deanhystad
  write to csv file problem jacksfrustration 11 1,372 Nov-09-2023, 01:56 PM
Last Post: deanhystad
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,308 Nov-09-2023, 10:56 AM
Last Post: mg24
  How do I read and write a binary file in Python? blackears 6 6,015 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Reading data from excel file –> process it >>then write to another excel output file Jennifer_Jone 0 1,046 Mar-14-2023, 07:59 PM
Last Post: Jennifer_Jone
  Read text file, modify it then write back Pavel_47 5 1,500 Feb-18-2023, 02:49 PM
Last Post: deanhystad
  how to read txt file, and write into excel with multiply sheet jacklee26 14 9,515 Jan-21-2023, 06:57 AM
Last Post: jacklee26
  How to write in text file - indented block Joni_Engr 4 6,362 Jul-18-2022, 09:09 AM
Last Post: Hathemand
  Upgrading from 2 to 3 and having file write problems KenHorse 2 1,430 May-08-2022, 09:47 PM
Last Post: KenHorse
  Cursor write 3rd file empty paulo79 3 1,826 Mar-10-2022, 02:51 PM
Last Post: DeaD_EyE

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020