Python Forum
Help with object interning in Python 3
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help with object interning in Python 3
#1


Just so we are on the same page, interning refers to the re-use or recycling of already created objects. So, if I ask for an object that has a particular signature already created, return that instead of creating a brand new one. In essence Python does this when it creates immutable objects.

I tried a few different methods for accomplishing this, but I decided on using metaclasses:

class MetaRegex(type):
    _instances = {}
    def __call__(cls, *args, **kwargs):
        key = cls.make_key(*args)

        if key in cls._instances:
            obj = cls._instances[key]
            obj.refcount += 1
        else:
            obj = cls._instances[key] = \
                super(MetaRegex, cls).__call__(*args, key=key, **kwargs)
        return obj
This works fine. It behaves as it's intended.

However, there are certain optimizations I perform within __new()__ of my derived classes. For example, if I'm creating an OR expression and either the left or right side is 0 (empty set), I can simplify the expression by returning either the left or right non-null child and skip creating this particular tree.

That works great except for Python's policy of:

https://docs.python.org/3/reference/data...ct.__new__

If __new__() returns an instance of cls, then the new instance’s __init()__ method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to __new__().

This behavior is BAD for me because if I return an already created object, it will call __init__() and proceed to screw up my object by re-initializing it's members, completely defeating the optimization.

To mitigate this, I would be forced to add silly protective code with hasattr(self, 'someattr') to protect all the initializers of my derived classes.

Is there a clean way to get around this? I can't be the only person wanting to do this kind of thing.
Reply


Messages In This Thread
Help with object interning in Python 3 - by clintolsen - Dec-19-2017, 12:47 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  python update binary object (override delivered Object properties) pierre38 4 1,872 May-19-2022, 07:52 AM
Last Post: pierre38
  want to know the kind of object whether its a python or json object johnkennykumar 5 74,459 Jan-21-2017, 08:47 AM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020