Feb-03-2023, 11:46 AM
Firstly, I have very little experience with python, so any constructive feedback on what I have done/should do with regards to the following issue that I have currently have, would be much appreciated.
I started creating a flask app that utilized both flask_sqlalchemy and flask_marshmallow. I got the code working and uploaded it to my shared hosting provider. All the rest api's are working as expected.
I now need to implement logic for processing large amounts of data in the database. After researching on-line, my plan was to use celery, and call a rest api that would then offload the job to celery so as not to block the rest api and cause a timeout. However, celery requires a worker server, such as RabbitMQ, and after speaking to the hosting company, they say this is not allowed.
I thought of creating another (non-flask) app that I can invoke from a cron job to do the heavy processing for me, which is allowed. The problem is that I don't want to duplicate the sqlalchemy database code that I already have, so my thought is to remove the dependency on flask for the sqlalchemy and marshmallow code and to turn all the database logic into a library that can be shared across both projects. This way, I would only have to write the database logic once.
So...
1) Am I heading down the right track?
2) Is there a better way to do this?
3) Should I just duplicate the database logic, so I can utilize the flask_sqlalchemy/flask_marshmallow packages in the flask app?
I started creating a flask app that utilized both flask_sqlalchemy and flask_marshmallow. I got the code working and uploaded it to my shared hosting provider. All the rest api's are working as expected.
I now need to implement logic for processing large amounts of data in the database. After researching on-line, my plan was to use celery, and call a rest api that would then offload the job to celery so as not to block the rest api and cause a timeout. However, celery requires a worker server, such as RabbitMQ, and after speaking to the hosting company, they say this is not allowed.
I thought of creating another (non-flask) app that I can invoke from a cron job to do the heavy processing for me, which is allowed. The problem is that I don't want to duplicate the sqlalchemy database code that I already have, so my thought is to remove the dependency on flask for the sqlalchemy and marshmallow code and to turn all the database logic into a library that can be shared across both projects. This way, I would only have to write the database logic once.
So...
1) Am I heading down the right track?
2) Is there a better way to do this?
3) Should I just duplicate the database logic, so I can utilize the flask_sqlalchemy/flask_marshmallow packages in the flask app?
