Python Forum
Is rememberme.memory() accurate for recursive structures?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Is rememberme.memory() accurate for recursive structures?
#3
(Dec-17-2020, 10:39 AM)Gribouillis Wrote: You could perhaps add a __slots__ member to your classes, especially the nodes and see if it changes anything (I don't know rememberme.memory()).

With slots:
Output:
BST BST, rememberme 3700 bytes BST getsizeof 40 bytes BST, rememberme 7460 bytes BST getsizeof 40 bytes values: 7 leaf node: 4354 BST, rememberme 7544 bytes BST getsizeof 40 bytes values: 3 7 leaf node: 4354 BST, rememberme 7628 bytes BST getsizeof 40 bytes values: 3 4 7 leaf node: 4354 BST, rememberme 7712 bytes BST getsizeof 40 bytes values: 3 4 7 9 List rememberme 232 bytes getsizeof 120 bytes values: [3, 4, 7, 9] Dictionary rememberme 344 bytes getsizeof 232 bytes values: {'a': 1, 'b': 2, 'c': 3, 'd': 4}
Without slots:
Output:
BST BST, rememberme 3991 bytes BST getsizeof 48 bytes BST, rememberme 7444 bytes BST getsizeof 48 bytes values: 7 leaf node: 4047 BST, rememberme 7624 bytes BST getsizeof 48 bytes values: 3 7 leaf node: 4047 BST, rememberme 7804 bytes BST getsizeof 48 bytes values: 3 4 7 leaf node: 4047 BST, rememberme 7984 bytes BST getsizeof 48 bytes values: 3 4 7 9 List rememberme 232 bytes getsizeof 120 bytes values: [3, 4, 7, 9] Dictionary rememberme 344 bytes getsizeof 232 bytes values: {'a': 1, 'b': 2, 'c': 3, 'd': 4}
According to rememberme, the slots provide about an 8% improvement, but that improvement diminishes and n increases.
Thanks for bringing that up, I was unaware of __slots__ and I enjoyed reading about them.
In that reading I came across the pympler module and its asizeof.asizeof() function. It provides what appears to be a more accurate size reading:
Output:
BST BST, rememberme 3700 bytes BST getsizeof 40 bytes BST asizeof 40 bytes BST, rememberme 7460 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 7 rememberme leaf node: 4354 asizeof leaf node: 104 BST, rememberme 7544 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 3 7 rememberme leaf node: 4354 asizeof leaf node: 104 BST, rememberme 7628 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 3 4 7 rememberme leaf node: 4354 asizeof leaf node: 104 BST, rememberme 7712 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 3 4 7 9 List rememberme 232 bytes getsizeof 120 bytes asizeof 248 bytes values: [3, 4, 7, 9] Dictionary rememberme 344 bytes getsizeof 232 bytes asizeof 584 bytes values: {'a': 1, 'b': 2, 'c': 3, 'd': 4}
The only issue with asizeof is that it does not look like it works with slots. I suppose that means it's completely dependent on the object's __dict__ or I messed up using it.

Output:
BST BST, rememberme 3700 bytes BST getsizeof 40 bytes BST asizeof 40 bytes BST, rememberme 7460 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 7 rememberme leaf node: 4354 asizeof leaf node: 104 BST, rememberme 7544 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 3 7 rememberme leaf node: 4354 asizeof leaf node: 104 BST, rememberme 7628 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 3 4 7 rememberme leaf node: 4354 asizeof leaf node: 104 BST, rememberme 7712 bytes BST getsizeof 40 bytes BST asizeof 40 bytes values: 3 4 7 9 List rememberme 232 bytes getsizeof 120 bytes asizeof 248 bytes values: [3, 4, 7, 9] Dictionary rememberme 344 bytes getsizeof 232 bytes asizeof 584 bytes values: {'a': 1, 'b': 2, 'c': 3, 'd': 4}
I'm going to read some more and maybe post it as a bug report on the author's git

edit:
It works with slots, I just needed to make some adjustments
Reply


Messages In This Thread
RE: Is rememberme.memory() accurate for recursive structures? - by Clunk_Head - Dec-17-2020, 06:07 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Cant's get accurate unbiased variance. Alex009988 1 2,315 Aug-13-2019, 02:55 PM
Last Post: ichabod801

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020