Configure Python Advanced Logging

Question

I would like to use logging for my modules, but I'm not sure how to develop the following requirements:

• normal levels of logging (information, error, warning, debugging), as well as some additional more detailed debugging levels
• registration messages can be of different types; some are for the developer, some are for the user; these types go to different outputs.
• errors should go to stderr
• I also need to keep track of which line of the module / function / code wrote the debug message so that I can activate or deactivate individual debug messages in the configuration
• I need to keep track of if errors have occurred at all, in order to eventually execute sys.exit() at the end of the program
• all messages should go to stdout until loggers are installed

I read the logging documentation, but I'm not sure the easiest way to use the logging module with the above requirements (how to use the concept of Logger, Handler, Filter, ...). Can you point out an idea to fix this? (for example, write a module with two users "logger", "developer", get from Logger ; make getLogger(__name__) ; contain an error flag, for example, ... etc.)

Answer 1

1) Adding more detailed debugging levels.

Do you find it?

Take a look at what the doc says:

Defining your own levels is possible, but not required , as existing levels have been selected based on practical experience. However, if you are sure that you need user levels, you should be very careful with , and it may be a very bad idea to define user levels if you are developing a library . This is because if several authors of the library define their own user levels, it is likely that the output of the journal from such several libraries used together will be difficult for the developer to control and / or interpret, since this numerical value may mean different things for different libraries.

Take a look also at When to use the log , there are two very good tables explaining when to use what.

In any case, if you think you need these additional logging levels, see: logging.addLevelName() .

2) Some logging messages for the developer, and some for the user

Use different log families with different handlers. In the database of each family, set Logger.propagate to False .

3) Errors should go to stderr

This already happens by default with StreamHandler :

class logging.StreamHandler (stream = no)

Returns a new instance of the StreamHandler class. If a stream is specified, the instance will use it to display the log; otherwise sys.stderr will be used.

4) Keep track of the source of the log message

Get registrars with different names, and in Formatter use format strings with %(name)s .

5) All messages should go to stdout until registrars are installed

Setting up your logging system should be one of the first things to do, so I really don't understand what that means. If you need to send messages to stdout, use print , as it should, and is already explained in When to use logging .

One final tip: read the magazine magazine carefully, as it covers pretty well what you need.

---

From the comment: How can I create a design for different sources, as well as filter my module?

I would not start filtering in the first place, filtering is difficult to maintain, and if they are all in one place, this place should contain too much information. Each module must set abd its own Logger (with its own handlers or filters), using or not its parent settings.

A very quick example:

 # at the very beginning root = logging.getLogger() fallback_handler = logging.StreamHandler(stream=sys.stdout) root.addHandler(fallback_handler) # first.py first_logger = logging.getLogger('first') first_logger.parent = False # ... set 'first' logger as you wish class Foo: def __init__(self): self.logger = logging.getLogger('first.Foo') def baz(self): self.logger.info("I'm in baz") # second.py second_logger = logging.getLogger('first.second') # to use the same settings # third.py abstract_logger = logging.getLogger('abs') abstract_logger.parent = False # ... set 'abs' logger third_logger = logging.getLogger('abs.third') # ... set 'abs.third' particular settings # fourth.py fourth_logger = logging.getLogger('abs.fourth') # [...]