Python Forum
Inspect.getmembers with isclass returns an empty list
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Inspect.getmembers with isclass returns an empty list
#1
Hello all!

I am awfully new to this site, but my lack of ideas has brought me here in hope that I might get help in here.

I am making an app, that is divided up into plugable modules that make up its functionality.

The general flow is like this: List a pre-defined directory, looking for any .py files and store those in a list.

Loop over the list and, using the importlib module (code snippet later), load the files into memory as python modules.

Using the inspect library, get all classes from the module object, returned in the last step, check if the class is a sub-class of one specifically pre-defined abstract class.

If all of those checks pass, instance the class and proceed with further functionality.

I have had all that working already, in the version of the app I made on Python 2, with the only difference being that instead of Importlib, I used the, now deprecated, imp library.

After upgrading to Python 3.7 and replacing imp with importlib, one key step broke - Inspect.getmembers now returns an empty list, instead of all the classes of the object. The code goes like this:

import importlib.util
import logging
import inspect
import errno
import sys
import os

    module_source_code_paths = []
    module_instances =[]
    module_inmemory_handles = []

        self.logger = logging.getLogger(os.uname()[1]+"_addomain")

        # Create handlers that the logging library will use. Here, we create two handlers
        # c_handler logs to the console (stdout), while f_handler logs to the log file in /var/log/addomain.log
        c_handler = logging.StreamHandler(stream=sys.stdout)
        if self.arguments.debug:
            f_handler = logging.FileHandler("./addomain.log", "a")
        else:
            f_handler = logging.FileHandler("/var/log/addomain.log", "a")

        # Set desired logging levels
        if self.arguments.verbose:
            self.logger.setLevel(logging.DEBUG)
            c_handler.setLevel(logging.DEBUG)
            f_handler.setLevel(logging.DEBUG)
        else:
            self.logger.setLevel(logging.INFO)
            c_handler.setLevel(logging.INFO)
            f_handler.setLevel(logging.INFO)

        # Set the appropriate log format strings
        if self.arguments.debug:
            c_format = logging.Formatter("[%(filename)s:%(lineno)s] - %(levelname)s - %(message)s")
        else:
            c_format = logging.Formatter("%(message)s")
        f_format = logging.Formatter("[%(asctime)s] [%(filename)s:%(lineno)s] [%(levelname)s]: %(message)s")

        # Add the formatters to their handlers
        c_handler.setFormatter(c_format)
        f_handler.setFormatter(f_format)

        # Finally, add both handlers to the main logging object
        self.logger.addHandler(c_handler)
        self.logger.addHandler(f_handler)

#First, we get all the python files in a pre-defined directory
        for (dirpath, dirnames, filenames) in os.walk(self.settings["MODULES_DIRECTORY"]):
            for filename in filenames:
                filename, file_extension = os.path.splitext(dirpath+filename)
                # And if it is a python file (or appears to be one...), then add it to the module_paths list
                # Used further to load them into memory and attempt to initialize their core class
                if file_extension == ".py":
                    self.logger.debug("Found file "+filename+file_extension)
                    self.module_source_code_paths.append(filename + file_extension)

#Then we try to load those files into memory as modules, using the importlib.util.spec_from_file and importlib.util.module_from_spec functions, and save the returned object to another list
        for module in self.module_source_code_paths:
            dirpath, filename = os.path.split(module)
            modname = os.path.splitext(module)[0]
            spec = importlib.util.spec_from_file_location(modname, dirpath+filename)
            self.module_inmemory_handles.append(importlib.util.module_from_spec(spec))

#Finally, we loop over the module objects and try to get a list of classes inside them, that we then check against several conditions, and save to another array
        for modobj in self.module_inmemory_handles:
            self.logger.debug(inspect.ismodule(modobj))
            classes = inspect.getmembers(modobj, inspect.isclass) #This line is the issue, as this function used to return a list of classes from the module in Python 2, but in Python 3 returns an empty list
An example of a valid plugin file might be the following:

from addomain import AddomainModuleBase
import errno
import os
import re

class ConfFilter(AddomainModuleBase):

    #Thank you, stack overflow person, for this regex. Source: https://stackoverflow.com/a/20204811
    __domain_regex = re.compile("(?=^.{4,253}$)(^((?!-)[a-zA-Z0-9-]{1,63}(?<!-)\.)+[a-zA-Z]{2,63}$)", re.IGNORECASE)

    def __init__(self, settings, arguments):
        if re.match(self.__domain_regex, arguments.domain) is None:
            raise NameError("Invalid domain name!")

        if "HOMEDIR" in settings:
            if not os.path.isdir(settings["HOMEDIR"]):
                raise OSError(errno.ENOENT, os.strerror(errno.ENOENT))

    def run(self):
        pass

    def cleanup(self):
        pass
The entire code has been taken out of context, in snippets, so please, ignore the details like a lack of proper function / class definition. Each block of code resides in its own function, which in turn, is in a common class.

I already tried asking at StackOverflow, but received no responses, so I am now turning to this community. I am open to even bigger code modifications, if the idea stayed the same.
Reply
#2
Alright, I managed to solve the issue.

Turns out what I got by the method described in the post wasn't exactly a module. It was... I am not even sure what. But by changing the loading routine as follows, it again started working as intended:

#spec = importlib.util.spec_from_file_location(modname, filename) We replace this step with the following
            loader = importlib.machinery.SourceFileLoader(filename, dirpath+"/"+filename)
            spec = importlib.util.spec_from_loader(loader.name, loader)
            mod = importlib.util.module_from_spec(spec)
            loader.exec_module(mod)
            self.module_inmemory_handles.append(mod)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Code with empty list not executing adeana 9 3,636 Dec-11-2023, 08:27 AM
Last Post: buran
  function returns dataframe as list harum 2 1,337 Aug-13-2022, 08:27 PM
Last Post: rob101
  set.difference of two list gives empty result wardancer84 4 1,434 Jun-14-2022, 01:36 PM
Last Post: wardancer84
  displaying empty list vlearner 5 1,606 Jan-19-2022, 09:12 AM
Last Post: perfringo
  Remove empty keys in a python list python_student 7 2,902 Jan-12-2022, 10:23 PM
Last Post: python_student
  Inspecting without using inspect deanhystad 3 1,544 Nov-11-2021, 07:17 PM
Last Post: snippsat
  function that returns a list of dictionaries nostradamus64 2 1,700 May-06-2021, 09:58 PM
Last Post: nostradamus64
  What is the value after JOINING an empty list? JaneTan 2 5,062 Jan-04-2021, 06:25 PM
Last Post: deanhystad
  Printing empty list? hhydration 2 2,078 Oct-28-2020, 11:34 AM
Last Post: Atekka
  Stop a function if the list it needs is empty Pedroski55 2 2,867 Jul-25-2020, 11:50 PM
Last Post: Pedroski55

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020