Python logging ensures that a handler will be added only once

Question

I have a piece of code that initializes the logger, as shown below.

logger = logging.getLogger() hdlr = logging.FileHandler('logfile.log') formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s') hdlr.setFormatter(formatter) logger.addHandler(hdlr) logger.setLevel(logging.DEBUG) 

Unfortunately, this code is called several times, is there a way to check if the handler really already exists - I would prefer to implement this without using Singleton.

EDIT: Sorry, forgot to mention this on python 2.5 - cheers, Richard

Answer 1

As @offbyone comments, you can add redundant handlers to a single registrar instance. python docs for logging say -

"Multiple calls to getLogger () with the same name return a link to the same log object."

Therefore, we do not need to worry about making the implementation singletonic, as it already is.

Unfortunately, for handlers associated with the same registrar instance, the same is not true . There may be duplicate handlers.

Example -

• Copy this code and save it in main.py

   import logging print 'inside main.py', print '-'*50 def logger(): print 'initializing logger....' logPath = '.' fileName = 'temp' # configure log formatter logFormatter = logging.Formatter("%(asctime)s [%(filename)s] [%(funcName)s] [%(levelname)s] [%(lineno)d] %(message)s") # configure file handler fileHandler = logging.FileHandler("{0}/{1}.log".format(logPath, fileName)) fileHandler.setFormatter(logFormatter) # configure stream handler consoleHandler = logging.StreamHandler() consoleHandler.setFormatter(logFormatter) # get the logger instance logger = logging.getLogger(__name__) # set the logging level logger.setLevel(logging.DEBUG) print 'adding handlers- ' #if not len(logger.handlers): logger.addHandler(fileHandler) logger.addHandler(consoleHandler) print 'logger initialized....\n' print 'associated handlers - ', len(logger.handlers) for handler in logger.handlers: print handler print return logger main_logger = logger() main_logger.info('utilizing main.py logger.') print 'exiting main.py', print '-'*50 

• and the following code in sub.py

   print 'inside sub.py', print '-'*50 print 'importing main.py' import main print 'imported main.py' import logging print 'getting logger instance in sub' sub_logger = main.logger() print 'got logger instance in sub' sub_logger.info("utilizing sub_logger") print 'exiting sub.py', print '-'*50 

• Run sub.py

   narayan@y510p:~/code/so$ python sub.py inside sub.py -------------------------------------------------- importing main.py inside main.py -------------------------------------------------- initializing logger.... adding handlers- logger initialized.... associated handlers - 2 <logging.FileHandler object at 0x7f7158740c90> <logging.StreamHandler object at 0x7f7158710b10> 2015-08-04 07:41:01,824 [main.py] [<module>] [INFO] [41] utilizing main.py logger. exiting main.py -------------------------------------------------- imported main.py getting logger instance in sub initializing logger.... adding handlers- logger initialized.... associated handlers - 4 # <===== 4 handlers (duplicates added) <logging.FileHandler object at 0x7f7158740c90> <logging.StreamHandler object at 0x7f7158710b10> <logging.FileHandler object at 0x7f7158710bd0> <logging.StreamHandler object at 0x7f7158710c10> got logger instance in sub 2015-08-04 07:41:01,824 [sub.py] [<module>] [INFO] [10] utilizing sub_logger 2015-08-04 07:41:01,824 [sub.py] [<module>] [INFO] [10] utilizing sub_logger exiting sub.py -------------------------------------------------- 

Consequently, multiple method calls returning the same log have added duplicate handlers.

Now, for your question -

is there any way to check if a handler already exists

Yes there is -

logger.handlers returns a list of all handlers associated with this logger .

Before adding handlers to the registrar instance, do not add duplicate handlers In main.py, just do not comment on the line labeled if not len(logger.handlers): and indent the following two lines:

 if not len(logger.handlers): logger.addHandler(fileHandler) logger.addHandler(consoleHandler) 

Now run sub.py again

 narayan@y510p:~/code/so$ python sub.py inside sub.py -------------------------------------------------- importing main.py inside main.py -------------------------------------------------- initializing logger.... adding handlers- logger initialized.... associated handlers - 2 <logging.FileHandler object at 0x7fd67a891c90> <logging.StreamHandler object at 0x7fd67a862b10> 2015-08-04 08:14:45,620 [main.py] [<module>] [INFO] [41] utilizing main.py logger. exiting main.py -------------------------------------------------- imported main.py getting logger instance in sub initializing logger.... adding handlers- logger initialized.... associated handlers - 2 # <===== Still 2 handlers (no duplicates) <logging.FileHandler object at 0x7fd67a891c90> <logging.StreamHandler object at 0x7fd67a862b10> got logger instance in sub 2015-08-04 08:14:45,620 [sub.py] [<module>] [INFO] [10] utilizing sub_logger exiting sub.py -------------------------------------------------- 

Also, if you want to limit the type of handlers that are added to the log instance, you can do something like this -

  print 'adding handlers- ' # allows to add only one instance of file handler and stream handler if len(logger.handlers) > 0: print 'making sure we do not add duplicate handlers' for handler in logger.handlers: # add the handlers to the logger # makes sure no duplicate handlers are added if not isinstance(handler, logging.FileHandler) and not isinstance(handler, logging.StreamHandler): logger.addHandler(fileHandler) print 'added file handler' logger.addHandler(consoleHandler) print 'added stream handler' else: logger.addHandler(fileHandler) logger.addHandler(consoleHandler) print 'added handlers for the first time' 

Hope this helps!

Edit:

Unfortunately, the same is not true for those associated with handlers with the same registrar instance. Duplicate handlers may be attached.

It turns out that the above statement is not entirely true.

Suppose we created and configured a logger called "main_logger" in the main module (which simply configures the logger, returns nothing).

 # get the logger instance logger = logging.getLogger("main_logger") # configuration follows ... 

Now in the submodule, if we create a child logger after the naming hierarchy "main_logger.sub_module_logger", we do not need to configure it in the submodule. Simply create a registrar for the naming hierarchy.

 # get the logger instance logger = logging.getLogger("main_logger.sub_module_logger") # no configuration needed # it inherits the configuration from the parent logger ... 

And he will not add a duplicate handler.

Link- Using Logging in Multiple Modules

Answer 2

Well, logger.addHandler () will not add a handler if the handler already exists. To check if a handler exists, you can check the logger.handlers list:

 logger = logging.getLogger() hdlr = logging.FileHandler('logfile.log') formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s') hdlr.setFormatter(formatter) logger.addHandler(hdlr) logger.setLevel(logging.DEBUG) print logger.handlers # [<logging.FileHandler object at 0x14542d0>] logger.addHandler(hdlr) print logger.handlers # [<logging.FileHandler object at 0x14542d0>] 

In addition, I suggest putting this code in your main () function if you have one or in your package __init__.py so that it does not call it every time. I also suggest using a named logger and not using the root log. Something like that:

 logger = logging.getLogger(__name__) ... 

Hope this was helpful :)

Answer 3

You can also just check if the list of handlers is empty. Here is the solution I came across:

 def setup_logging(self, logfile): self._logger = logging.getLogger('TestSuite') self._logger.setLevel(logging.INFO) host = socket.gethostname().split('.')[0] if self._logger.handlers == []: fh = logging.handlers.RotatingFileHandler(logfile, maxBytes=10*1024*1024, backupCount=5) strfmt = "%" + "(asctime)s [%s] " % host + "%" + "(message)s" fmt = logging.Formatter(strfmt, datefmt="%Y.%m%d %H:%M:%S") fh.setFormatter(fmt) self._logger.addHandler(fh) self._logger.info('-' * (55 - len(host))) 

I saw how the handler was added several times, so each log message was written to the log file more than once, and this fixed it.

Answer 4

Try checking if logger installed. For example, if this code is inside a function:

 logger = None def init_logger(): global logger if logger is not None: #logger has already been initialized return logger = logging.getLogger() hdlr = logging.FileHandler('logfile.log') formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s') hdlr.setFormatter(formatter) logger.addHandler(hdlr) logger.setLevel(logging.DEBUG)