SQL Alchemy dynamic class - declarative_base losing attributes

SQL Alchemy dynamic class - declarative_base losing attributes - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: SQL Alchemy dynamic class - declarative_base losing attributes (/thread-23571.html)

SQL Alchemy dynamic class - declarative_base losing attributes - mrdominikku - Jan-06-2020

Hi,

Based on below posts
- dynamic-schema
- Dynamic Class Creation in SQLAlchemy

I wanted to make similar thing in my project. I use declarative_base class for core ORM models stored in db_models.py, additionally depends on user settings I want to create and store dynamic classes in BASE._dec_class_registry (db_manage.py>LotteryDatabase>_initial_database>create_class_model) those dynamic classes vary depends on params combinations from games_config.py.
My problem is when the first create_class_model method is called it's correctly adding class to BASE._dec_class_registry, but when calling next one BASE "losing" first class and adding new one and so on. When BASE.metadata.create_all() is called all tables are created in schema, but I can't access this dynamic classes using LotteryDatabase>set_model_by_table_name.

I tried doing it with raw test example:

models.py

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Integer, Column

BASE = declarative_base()


class NewClass(BASE):

    __tablename__ = 'test'

    id = Column('id', Integer, primary_key=True)

test.py

from models import *
from sqlalchemy import create_engine

DB_ENGINE = 'sqlite:///my_db.db'


def main():

    engine = create_engine(DB_ENGINE)

    BASE.metadata.create_all(engine)

    input_params = {'__tablename__': 'test1',
                    'id': Column('id', Integer, primary_key=True)}

    _ = type('InputClass', (BASE,), input_params)

    model_params = {'__tablename__': 'test2',
                    'id': Column('id', Integer, primary_key=True)}

    _ = type('ModelClass', (BASE,), model_params)

    predict_params = {'__tablename__': 'test3',
                      'id': Column('id', Integer, primary_key=True)}

    _ = type('PredictClass', (BASE,), predict_params)

    BASE.metadata.create_all(engine)

All classes where created and added to BASE, so problem is with my code and I am missing something. Your fresh look would be very helpful!
Application is fully operational and including requirements.txt file for packages.

For better inside what I am talking about, step by step example using debug mode. (look image below)

I have rewrote code so all class attributes are stored in dictionary and then used in creating models.

-->> [Image: view?usp=sharing]

RE: SQL Alchemy dynamic class - declarative_base losing attributes - Larz60+ - Jan-07-2020

you might find this useful: https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database
also: https://docs.pylonsproject.org/projects/akhet/en/v1.0.1/model_examples.html#common-base-class

RE: SQL Alchemy dynamic class - declarative_base losing attributes - LeanbridgeTech - Jan-07-2020

your sample code puts the classes in _decl_class_registry fine, below is your code with the classes being retrieved:

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Integer, Column

BASE = declarative_base()

class NewClass(BASE):

__tablename__ = 'test'

id = Column('id', Integer, primary_key=True)

input_params = {'__tablename__': 'test1',
'id': Column('id', Integer, primary_key=True)}

_ = type('InputClass', (BASE,), input_params)

model_params = {'__tablename__': 'test2',
'id': Column('id', Integer, primary_key=True)}

_ = type('ModelClass', (BASE,), model_params)

predict_params = {'__tablename__': 'test3',
'id': Column('id', Integer, primary_key=True)}

_ = type('PredictClass', (BASE,), predict_params)

input_class = BASE._decl_class_registry["InputClass"]
model_class = BASE._decl_class_registry["ModelClass"]
predict_class = BASE._decl_class_registry["PredictClass"]

from sqlalchemy.orm import Session
print(Session().query(predict_class))
I don't have the resources to go looking at your full application but the basic idea works, if you need help with your app see if you can find folks on the IRC channel to take a look for you, or otherwise try to reverse your code back to the sample above to see what's different.

RE: SQL Alchemy dynamic class - declarative_base losing attributes - mrdominikku - Jan-07-2020

@LeanbridgeTech

Thanks for that! We are looking at this with SQL ALchemy team to resolve that. Once we find solution will post it here!

RE: SQL Alchemy dynamic class - declarative_base losing attributes - mrdominikku - Jan-10-2020

SQL Alchemy team found problem. BASE._dec_class_registry losing attributes due to multithreading. As a solution for that I create global DYNAMIC_CLASS and store all _ = type(class_name, (BASE,), params) and call it on run. BASE._dec_class_registry don't losing attributes then