Announcing the configoose module!

[Gribouillis]

I uploaded a new module in Github, it is named « configoose ». It is not yet in Pypi and it is at an early stage of its life, but you can already install it as described on the Github page.

It is not yet documented, but I'll give you here an idea and a small example of what it does (changed! documentation now at https://configoose.readthedocs.io/en/latest/index.html)

I wrote this module because when I write a program that needs configuration files, I want to be free as a user to store these files anywhere in my file system, and perhaps in more exotic places such as inside a zip file or a database or on the web etc.

The problem is: how can the program or the module find its configuration files if they can be anywhere on the file system?

The solution is: the program uses an abstract 'address' for the configuration file and it asks someone to find the configuration
file registered with that address. This someone is the configoose module.

Here is a small example of how it works with a program configured with configparser (this is called a protocol in configoose's jargon)

I first create a configuration file for the configparser protocol by running the command

Output:
python -m configoose template configoose.protocol.configparser.Protocol > paillasse/tmp/french.cfg

Now I edit the configuration file and populate it

Output:
{
    "address" : "s8dzw5y5anduiodmrdrxdgxaa",
    "protopath" : "configoose.protocol.configparser.Protocol",
}
[spam]
ham =
    slice A
    slice B
    slice C

[more]
eggs = 1000

It is a normal configparser file except that there is a small python dictionary at the beginning called a « preamble ». I don't touch this preamble.

Now I register my configuration file in the configoose system so that other modules can find it

Output:
python -m configoose moor -a s8dzw5y5anduiodmrdrxdgxaa initial paillasse/tmp/french.cfg

It remains to write a module that uses this configuration. Here is one

from configoose import Configurator

cfg = Configurator("s8dzw5y5anduiodmrdrxdgxaa")

@cfg.add_protocol("configoose.protocol.configparser.Protocol")
def handler(ap, preamble, parser):
    s = parser["spam"]["ham"]
    ham = [y.strip() for y in s.strip().split("\n") if y.strip()]
    print('ham = ', ham)
    print(f"There are {parser['more']['eggs']} eggs!")

cfg.run()

Here the handler function receives a ConfigParser instance among other arguments, which read the configuration file. Here is the output

Output:
λ python paillasse/tmp/frenchman.py
ham =  ['slice A', 'slice B', 'slice C']
There are 1000 eggs!

There are a huge amount of details to be explained about configoose. It is an infinitely extensible module. Arbitrary configuration protocols can be defined (3 are included by default) and there is potentially no limit as to where the configuration files can be stored.

The next step for me in the next days is to develop the basic documentation.

Play with the module, enjoy and post feedback!

[snippsat]

Good idea,when i test on Windows and also on Linux(where i have Python installed bye pyenv).
There is Paths problems to where it find Python version used,on Windows i did tests with virtual environment(venv).

Output:
G:\div_code\goose_env
(goose_env) λ python -m configoose conf --marina G:\div_code\goose_env
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "G:\div_code\goose_env\Lib\site-packages\configoose\__main__.py", line 210, in <module>
    main(sys.argv[1:])
  File "G:\div_code\goose_env\Lib\site-packages\configoose\util\algorithm.py", line 21, in __call__
    return instance.run(*args)
           ^^^^^^^^^^^^^^^^^^^
  File "G:\div_code\goose_env\Lib\site-packages\configoose\__main__.py", line 79, in run
    callback(command, args[1:])
  File "G:\div_code\goose_env\Lib\site-packages\configoose\cli\subcommand\conf.py", line 71, in main
    dest.write_text(code)
  File "C:\python312\Lib\pathlib.py", line 1047, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python312\Lib\pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Tom\\AppData\\Roaming\\Python\\Python312\\site-packages\\configooseconf.py'  

So it goes out of venv and try find path to Python(i have never used .....Roaming\\Python\\Python312\\site-packages) as path.
My default root Python is(C:\Python312) and virtual environment Path used now is G:\div_code\goose_env\Scripts\python.

On Linux where i use pyenv it guess wrong Path to Python version used.
I think your approach where it need to find path to user Python version can be tricky🔨 to get right.
Example if use configparser it work fine in a venv,as it only care about the path to eg example.ini file.

[Gribouillis]

So it goes out of venv and try find path to Python(i have never used .....Roaming\\Python\\Python312\\site-packages) as path.

The problem is I don't know exactly how user site-packages directory is handled in the case of virtual envs. What you could do in the meantime is to pass the option --dest /path/to/your/venv/site-packages to this command line.

Alternatively you can try the --global option. I think it should install a configooseglobalconf.py in your virtual environment's site-packages directory.

I'll try to find a solution that works in all cases, but really I'm not knowlegeable concerning virtual environments.

EDIT:
I tried with pyenv in Linux and the --global option worked

Output:
python -m configoose conf --global --marina ~/tmp

>>> import configooseglobalconf
>>> configooseglobalconf
<module 'configooseglobalconf' from '/home/eric/.pyenv/versions/3.12.4/lib/python3.12/site-packages/configooseglobalconf.py'>
>>> 

With this setup, I can moor a configuration file in the 'initialglobal' marina instead of 'initial'

Output:
python -m configoose moor initialglobal paillasse/tmp/french.cfg

EDIT 2
I tried again with the --global option in a pyenv virtualenv and it worked also

Output:
(somevirtualenv) λ python -m configoose conf --global --marina ~/tmp

(somevirtualenv) λ python
Python 3.12.4 (main, Jun 29 2024, 11:19:46) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import configooseglobalconf
>>> configooseglobalconf
<module 'configooseglobalconf' from '/home/eric/.pyenv/versions/somevirtualenv/lib/python3.12/site-packages/configooseglobalconf.py'>
>>> 

Output:
(somevirtualenv) λ python -m configoose moor initialglobal paillasse/tmp/french.cfg
(somevirtualenv) λ 
(somevirtualenv) λ python -m configoose find s8dzw5y5anduiodmrdrxdgxaa
FileInOsMediator(PosixPath('/home/eric/Projets/Scratch/2023-01/paillasse/tmp/french.cfg'))

EDIT 3:
If you don't like the word 'initialglobal', you can still edit the file configooseglobalconf.py and add tags to your first marina, for example

{
    "address": "configooseglobalconf-address",
    'protopath': "configoose.protocol.methodic.Protocol",
}
__doc__ = """configooseglobalconf.py

This is configoose's root database configuration module.
The basic configuration is to add marinas, usually OS directories,
where configuration mediators can be stored.
"""

def configure(handler):
    handler.add_marina(
        path='/home/eric/tmp',
        style="os-directory",
        tags={'initialglobal', 'bönhamn'},
    )

Now you can moor your configuration file in the Bönhamn marina

Output:
(somevirtualenv) λ python -m configoose moor bönhamn paillasse/tmp/french.cfg

[Gribouillis]

I just checked out your configoose module on GitHub and it looks super interesting!

Thank you for you interest in configoose. I uploaded a new version with a Sphinx documentation which is very much a work in progress. Unfortunately, I was unable to build this doc on readthedocs for an unknown reason. You can currently build the documentation locally with

Output:
python -m pip install sphinx
python -m pip install sphinx-rtd-scheme
python -m pip install marshmallow

then navigate to the docs/ directory and type

Output:
make clean && make html

Hopefully I'll be able to solve the problem with readthedocs next week.

@snippsat In the new version I changed things a little bit with regards to your issues. I made the global install of configooseconf.py the default, which should work well in virtualenv, and I added an optional installation of a userconfigooseconf.py by passing a --user option to the conf command

Output:
λ python -m configoose conf -h
usage: python -m configoose conf [-h] [-u] -m MARINADIR [-d [DESTDIR]]

Create file configooseconf.py or userconfigooseconf.py

options:
  -h, --help            show this help message and exit
  -u, --user            create user configuration file
  -m MARINADIR, --marina MARINADIR
                        directory to use as initial marina
  -d [DESTDIR], --dest [DESTDIR]
                        directory on the python path to write the python module, if value
                        is - the module is written to stdout. If omitted, module is
                        written to site-packages directory, global or user depending on
                        the --user option.

[Gribouillis]

I finally managed to make it work on readthedocs. The documentation can be found at https://configoose.readthedocs.io/en/latest/index.html
This documentation is a work in progress: more will come soon, especially documentation about how to extend configoose to store configuration files in unconventional places and also about new «protocols» that can easily be defined. Currently I'm concentrating on improving the code's docstrings for the autodoc documentation. Stay tuned! I'll post news and examples later.