Hi everyone! How do you organize your code? I like to separate everything into directories, preferably in the form of a monorepo. Custom libraries, services, etc. I then import everything using Poetry. An alternative is to keep either all scripts in one directory or in directories with the main script. I don't like this way of organizing, but unfortunately, my method (monorepo) causes a significant increase in memory usage. Each library and service has its own virtual environment, and many dependencies are repeated. Isn't there a better way?
I fail to see why one method requires a virtual environment for each library/service, but the other does not. Can you explain why you think this is the case? Maybe you are wrong.
I don't think monorepo is a good way to organize code. I find them difficult to work with. But I don't see why a monorepo would not work for Python or why it would be any more inefficient than monorepos normally are.
I think we might have misunderstood each other. I'm not saying that other approaches don't require a virtual environment or that they can't be used. I'm also not claiming that a monorepo doesn't work well for Python or that it's more or less efficient.
I'm simply asserting that if I use a monorepo and import my own libraries into other projects/services, I have dependencies redundancy.
Alternatively, I could create a simple directory structure like my_project/modules and my_project/main.py or app.py and import my modules in the app. But if I do it this way, and I have several independent sub-projects/services, each utilizing a script (with the same content) in modules, then I have code redundancy. I need to place the same code in each modules directory.
I hope this explanation is clear.
And one more thing - I assume upfront that I'm omitting something obvious. Blindsight ;p That's why I'm writing this here.
FYI: I never start a bona fide project without first creating a virtual environment.
One of the main reasons is that each project can have it's own version of python.
There are many other benefits, such as keeping packages localized, and requirements to a minimum.