Conditionally lazy load modules

[amb85]

I want to use a decorator to append a function to a list of commands to be run. I'd like to be able to use the decorator in arbitrary modules and not explicitly have to import the module. i.e. use the decorator wherever and I'll sort out the logic of finding the relevant files and loading them.

So far, I'm using glob to find all files, filtering the list of results to obtain the possible candidate files, then opening each file and appending it to a list if it contains the name of my decorator. I'm then using importlib to load the modules, except that each applicable module is being loaded 3 times (I got this down to 2 by also checking the absolute file path).

FileInfo = namedtuple('FileInfo', ['name', 'path'])


loaded_validations = []


def _applicable(path):
    return (os.path.abspath(path) != __file__ and path != __file__) and 'test_' not in path


if len(loaded_validations) == 0:
    _files = []
    _contenders = [path for path in glob.glob('**/*.py', recursive=True) if _applicable(path)]
    for contender in _contenders:
        with open(contender, 'r', encoding='utf-8') as f:
            for line in f:
                if '@validation' in line:
                    _files.append(FileInfo(os.path.basename(contender).replace('.py', ''), contender))
                    break

    for file in _files:
        spec = importlib.util.spec_from_file_location(file.name, file.path)
        module = importlib.util.module_from_spec(spec)
        spec.loader.exec_module(module)


def validation(function, name, description):
    # Check for duplicate name, raise error if so, do stuff with name and description.
    loaded_validations.append(function)
    # return wrapped function.

Is there a better way to approach solving this problem?

[Gribouillis]

So you want to import all the files in a hierarchy that contain the @validation decorator.

A list of such files can be very easily generated by an external command, for example in linux

Output:
grep -l -E -e "@validation" -r --include "*.py" .

I would try to keep a list of these files in a separate file "validation-db" and have python read this file to know which files to import. The program could optionally call the command that generates the list, but the logic here is that the files are only edited now and then, so the list should remain stable.