Python: a logging setting that allows multi-line strings: logging.info ('foo \ nbar')

Question

So far I am doing simple logging in files, and if I write a multi-line string, the result is as follows:

Log output:

logging.info('foo\nbar') 

Logfile:

 2018-03-05 10:51:53 root.main +16: INFO [28302] foo 

bar

So far, all statements that do not contain "INFO" or "DEBUG" are reported to the statements.

This means that the string bar being reported. This is a false positive.

Environment: Linux.

How to set up logging in Python to keep INFO foo\nbar on one line and ignore the whole line, since this is only "INFO"?

Note. Yes, you can filter logging in the interpreter. Unfortunately, this is not a question. This question is different. Logging occurs first. Then the logs are analyzed.

Here is a script to reproduce it:

 import sys import logging def set_up_logging(level=logging.INFO): root_logger = logging.getLogger() root_logger.setLevel(level) handler = logging.StreamHandler(sys.stdout) handler.setFormatter( logging.Formatter('%(asctime)s %(name)s: %(levelname)-8s [%(process)d] %(message)s', '%Y-%m-%d %H:%M:%S')) root_logger.addHandler(handler) def main(): set_up_logging() logging.info('foo\nbar') if __name__ == '__main__': main() 

Thinking about it again, I think the real question is: what logging format is possible? Just deleting lines in messages that span multiple lines makes it difficult to read some data for the human eye. On the other hand, the current 1: 1 ratio between logging.info () and the line in the log file is easy to read .... I'm not sure

Answer 1

Usually I have a class to configure logging, but you can achieve what you want with custom logging.Formatter :

 import logging class NewLineFormatter(logging.Formatter): def __init__(self, fmt, datefmt=None): """ Init given the log line format and date format """ logging.Formatter.__init__(self, fmt, datefmt) def format(self, record): """ Override format function """ msg = logging.Formatter.format(self, record) if record.message != "": parts = msg.split(record.message) msg = msg.replace('\n', '\n' + parts[0]) return msg 

The format() function above separates the lines and replicates the timestamp / log-preamble in each line (after each \n )

Now you need to attach the formatter to the root log. In fact, you can bind it to any handler if you create your own setting / logging structure:

 # Basic config as usual logging.basicConfig(level=logging.DEBUG) # Some globals/consts DATEFORMAT = '%d-%m-%Y %H:%M:%S' LOGFORMAT = '%(asctime)s %(process)s %(levelname)-8s %(filename)15s-%(lineno)-4s: %(message)s' # Create a new formatter formatter = NewLineFormatter(LOGFORMAT, datefmt=DATEFORMAT) # Attach the formatter on the root logger lg = logging.getLogger() # This is a bit of a hack... might be a better way to do this lg.handlers[0].setFormatter(formatter) # test root logger lg.debug("Hello\nWorld") # test module logger + JSON lg = logging.getLogger("mylogger") lg.debug('{\n "a": "Hello",\n "b": "World2"\n}') 

The above gives you:

 05-03-2018 08:37:34 13065 DEBUG test_logger.py-47 : Hello 05-03-2018 08:37:34 13065 DEBUG test_logger.py-47 : World 05-03-2018 08:37:34 13065 DEBUG test_logger.py-51 : { 05-03-2018 08:37:34 13065 DEBUG test_logger.py-51 : "a": "Hello", 05-03-2018 08:37:34 13065 DEBUG test_logger.py-51 : "b": "World2" 05-03-2018 08:37:34 13065 DEBUG test_logger.py-51 : } 

Note that I am .handlers[0] to the .handlers[0] root registrar, which is a bit hacked, but I could not find a way around this. Also pay attention to JSON formatted printing :)

Answer 2

You can use:

 logging.basicConfig(level=your_level) 

where your_level is one of the following:

  'debug': logging.DEBUG, 'info': logging.INFO, 'warning': logging.WARNING, 'error': logging.ERROR, 'critical': logging.CRITICAL 

In your case, you can use a warning to ignore the information

 import logging logging.basicConfig(filename='example.log',level=logging.WARNING) logging.debug('This message should go to the log file') logging.info('So should this') logging.warning('And this, too') Output: WARNING:root:And this, too 

You can try disabling INFO before registering as

 import logging logging.basicConfig(filename='example.log') logging.debug('This message should go to the log file') logging.disable(logging.INFO) logging.info('So should this') logging.disable(logging.NOTSET) logging.warning('And this, too') Output: WARNING:root:And this, too 

or

 logger.disabled = True someOtherModule.function() logger.disabled = False 

Answer 3

I think that maintaining this ratio 1: 1, one line in the log file for each call to logging.info() , it is very desirable that the log files be simple and clear. Therefore, if you really need to write a newline character, then I just register the string representation, for example:

 logging.info(repr('foo\nbar')) 

Outputs:

 2018-03-05 11:34:54 root: INFO [32418] 'foo\nbar' 

A simple option is to record each part separately:

 log_string = 'foo\nbar' for item in log_string.split('\n'): logging.info(item) 

Outputs:

 2018-03-05 15:39:44 root: INFO [4196] foo 2018-03-05 15:39:44 root: INFO [4196] bar