+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Flask/non-flask database sharing (/thread-39344.html)
Flask/non-flask database sharing - MorganSamage - Feb-03-2023
Firstly, I have very little experience with python, so any constructive feedback on what I have done/should do with regards to the following issue that I have currently have, would be much appreciated.
I started creating a flask app that utilized both flask_sqlalchemy and flask_marshmallow. I got the code working and uploaded it to my shared hosting provider. All the rest api's are working as expected.
I now need to implement logic for processing large amounts of data in the database. After researching on-line, my plan was to use celery, and call a rest api that would then offload the job to celery so as not to block the rest api and cause a timeout. However, celery requires a worker server, such as RabbitMQ, and after speaking to the hosting company, they say this is not allowed.
I thought of creating another (non-flask) app that I can invoke from a cron job to do the heavy processing for me, which is allowed. The problem is that I don't want to duplicate the sqlalchemy database code that I already have, so my thought is to remove the dependency on flask for the sqlalchemy and marshmallow code and to turn all the database logic into a library that can be shared across both projects. This way, I would only have to write the database logic once.
So...
1) Am I heading down the right track?
2) Is there a better way to do this?
3) Should I just duplicate the database logic, so I can utilize the flask_sqlalchemy/flask_marshmallow packages in the flask app?
RE: Flask/non-flask database sharing - buran - Feb-03-2023
(Feb-03-2023, 11:46 AM)MorganSamage Wrote: However, celery requires a worker server, such as RabbitMQ, and after speaking to the hosting company, they say this is not allowed.It could be Redis as well if available from hosting
RE: Flask/non-flask database sharing - MorganSamage - Feb-03-2023
(Feb-03-2023, 11:56 AM)buran Wrote:(Feb-03-2023, 11:46 AM)MorganSamage Wrote: However, celery requires a worker server, such as RabbitMQ, and after speaking to the hosting company, they say this is not allowed.It could be Redis as well if available from hosting
Hi, and thanks for the quick response.
I did think of Redis, but the hosting company were implying that this type of functionality is not allowed, and I did send them the info on RabbitMQ and Redis, but just to make sure, I will ask them the direct question, regarding Redis.