Python Forum
How to move a class to a custom module?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to move a class to a custom module?
#1
Question 
I am using Python 3.6.0 :: Continuum Analytics, Inc. on Linux.
I have ContextFilter Class in my python script which I am using to have a counter variable in log formatter. It works as expected and prints value of incremented variable in log.

But when I try to move this Class to a custom module that I have created it always prints the initial value of variable and not the incremented value. Obviously I am missing something.

Contents of module log_format.py:
import logging
import datetime as dt

# As I need date in 2022-02-22 16:42:44.289193 format
class Formatter(logging.Formatter):
    converter=dt.datetime.fromtimestamp
    def formatTime(self, record, datefmt=None):
        ct = self.converter(record.created)
        if datefmt:
            s = ct.strftime(datefmt)
        else:
            t = ct.strftime("%Y-%m-%d %H:%M:%S")
            s = "%s,%03d" % (t, record.msecs)
        return s


#class ContextFilter(logging.Filter): # RUN 2
#    def __init__(self, i):           # RUN 2
#        self.i = i                   # RUN 2
#                                     # RUN 2
#    def filter(self, record):        # RUN 2
#        record.log_count = self.i    # RUN 2
#        return True                  # RUN 2
Contents of python script log_format_test.py
import os
import sys
import logging

py_lib = os.environ.get('app_dir') + "/lib"
sys.path.append(py_lib)

import log_format as lf

class ContextFilter(logging.Filter): # RUN 1
    def filter(self, record):        # RUN 1
        record.log_count = i         # RUN 1
        return True                  # RUN 1


logger = logging.getLogger(__name__)
l_level = 'INFO' #'DEBUG' #'INFO'
l_level = eval("logging." + l_level.upper())
logger.setLevel(l_level)

int_var = os.getpid()
i = 0
str_var = os.path.basename(__file__)

handler = logging.StreamHandler()
handler.setLevel(l_level)
lf_format = f'ZZZZZZZZ|%(asctime)s|{int_var}|%(log_count)s|{str_var}|%(message)s'
formatter = lf.Formatter(fmt=lf_format, datefmt='%Y-%m-%d %H:%M:%S.%f') # 2022-02-22 16:42:44.289193

logger.addFilter(ContextFilter()) # RUN 1
#logger.addFilter(lf.ContextFilter(i)) # RUN 2

handler.setFormatter(formatter)
logger.addHandler(handler)

i += 1; logger.info("I am INFO 0")
i += 1; logger.info("I am INFO 1")
i += 1; logger.debug("I am DEBUG 0")
When I run script log_format_test.py it does print log as expect with 4th field incremented for each log entry (RUN 1):
ZZZZZZZZ|2022-02-24 11:41:19.578733|20527|1|log_format_test.py|I am INFO 0
ZZZZZZZZ|2022-02-24 11:41:19.578868|20527|2|log_format_test.py|I am INFO 1
But when I try to move class ContextFilter out of log_format_test.py in to log_format.py by uncommenting # RUN 2 lines and commenting # RUN 1 lines, the log shows only initial value in 4th field which is 0 (RUN 2):
ZZZZZZZZ|2022-02-24 11:55:24.679430|29115|0|log_format_test.py|I am INFO 0
ZZZZZZZZ|2022-02-24 11:55:24.679552|29115|0|log_format_test.py|I am INFO 1
Question: How can I move move class ContextFilter out of log_format_test.py in to log_format.py (or a new module of it's own)? What am I missing here?
Reply
#2
I am confused. In our test you used the global variable "i" and you incremented it like htis:
i += 1; logger.info("I am INFO 0")
In log_format.py you changed "i" to an instance variable and you stop incrementing it.

Why do you expect "i" to change if you don't change it? Somewhere you will need an obj.i += 1 where obj is an instance of ContextFilter/
Reply
#3
(Feb-28-2022, 04:42 PM)deanhystad Wrote: I am confused. In our test you used the global variable "i" and you incremented it like htis:
i += 1; logger.info("I am INFO 0")
In log_format.py you changed "i" to an instance variable and you stop incrementing it.

Why do you expect "i" to change if you don't change it?

Thank you for your reply.
Actually in RUN 2 I did try to have class ContextFilter as
class ContextFilter(logging.Filter):
    def filter(self, record):
        record.log_count = i
        return True
And call the logger.addFilter as
logger.addFilter(my.ContextFilter())
but then I get following error:
Traceback (most recent call last):
  File "./log_format_test.py", line 36, in <module>
    i += 1; logger.info("I am INFO 0")
  File "~/miniconda3/lib/python3.6/logging/__init__.py", line 1301, in info
    self._log(INFO, msg, args, **kwargs)
  File "~miniconda3/lib/python3.6/logging/__init__.py", line 1437, in _log
    self.handle(record)
  File "~/miniconda3/lib/python3.6/logging/__init__.py", line 1446, in handle
    if (not self.disabled) and self.filter(record):
  File "~/miniconda3/lib/python3.6/logging/__init__.py", line 713, in filter
    result = f.filter(record)
  File "/local_home/log_format.py", line 19, in filter
    record.log_count = i
NameError: name 'i' is not defined
I am new to object oriented programming so don't understand it very well yet. I tried defining i as global in log_format_test.py and tried passing it as
logger.addFilter(my.ContextFilter(i))
and many other combinations. The problem that you pointed was one of those many attempts.

I think I am missing something very basic here.
Reply
#4
Where you used to do "i += 1" you'll now have "obj.i += 1" where obj is an instance of ContextFilter.
Reply
#5
Thank you @deanhystad for your helpful reply. Following your suggestion I was able to figure out how I could do this.

Contents of module log_format.py
    import logging
    import datetime as dt
    
    
    class Formatter(logging.Formatter):
        converter=dt.datetime.fromtimestamp
        def formatTime(self, record, datefmt=None):
            ct = self.converter(record.created)
            if datefmt:
                s = ct.strftime(datefmt)
            else:
                t = ct.strftime("%Y-%m-%d %H:%M:%S")
                s = "%s,%03d" % (t, record.msecs)
            return s
    
    
    class ContextFilter(logging.Filter): # RUN 2
        def __init__(self, i):           # RUN 2
            self.i = i                   # RUN 2
                                         # RUN 2
        def filter(self, record):        # RUN 2
            self.i += 1                  # NEW LINE
            record.log_count = self.i    # RUN 2
            return True                  # RUN 2
Contents of python script log_format_test.py
    import os
    import sys
    import logging
    
    py_lib = os.environ.get('app_dir') + "/lib"
    sys.path.append(py_lib)
    
    import log_format as lf
    
    
    logger = logging.getLogger(__name__)
    l_level = 'INFO' #'DEBUG' #'INFO'
    l_level = eval("logging." + l_level.upper())
    logger.setLevel(l_level)
    
    int_var = os.getpid()
    i = 0
    str_var = os.path.basename(__file__)
    
    handler = logging.StreamHandler()
    handler.setLevel(l_level)
    lf_format = f'ZZZZZZZZ|%(asctime)s|{int_var}|%(log_count)s|{str_var}|%(message)s'
    formatter = lf.Formatter(fmt=lf_format, datefmt='%Y-%m-%d %H:%M:%S.%f') # 2022-02-22 16:42:44.289193
    
    obj_filter = lf.ContextFilter(i)                    # NEW LINE
    logger.addFilter(lf.ContextFilter(obj_filter)) # CHANGED LINE
    
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    
    logger.info("I am INFO 1")
    logger.info("I am INFO 2")
    logger.debug("I am DEBUG 3")
This results in printing the log as expected and I don't even have to manually increment i every time.

    ZZZZZZZZ|2022-02-24 11:41:19.578733|20527|1|log_format_test.py|I am INFO 1
    ZZZZZZZZ|2022-02-24 11:41:19.578868|20527|2|log_format_test.py|I am INFO 2
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to read module/class from list of strings? popular_dog 1 424 Oct-04-2023, 03:08 PM
Last Post: deanhystad
  My code displays too much output when importing class from a module lil_e 4 1,103 Oct-22-2022, 12:56 AM
Last Post: Larz60+
  Error when refering to class defined in 'main' in an imported module HeRo 2 2,334 Apr-13-2021, 07:22 PM
Last Post: HeRo
  Custom file class deanhystad 11 4,266 Feb-01-2021, 05:09 PM
Last Post: nilamo
  importing same python library in multiple custom module escape_freedom13 6 3,724 May-10-2020, 07:01 PM
Last Post: escape_freedom13
  How to serialize custom class objects in JSON? Exsul1 4 3,432 Sep-23-2019, 08:27 AM
Last Post: wavic
  How can I save a class as a module sakawaka 2 4,856 Feb-07-2018, 03:01 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020