Can Python's logging format be modified depending on the message log level?

Python Problem Overview

I'm using Python's logging mechanism to print output to the screen. I could do this with print statements, but I want to allow a finer-tuned granularity for the user to disable certain types of output. I like the format printed for errors, but would prefer a simpler format when the output level is "info."

For example:

  logger.error("Running cmd failed")
  logger.info("Running cmd passed")

In this example, I would like the format of the error to be printed differently:

> # error > Aug 27, 2009 - ERROR: Running cmd failed > # info > Running cmd passed

Is it possible to have different formats for different log levels without having multiple logging objects? I'd prefer to do this without modifying the logger once it's created since there are a high number of if/else statements to determine how the output should be logged.

Python Solutions

Solution 1 - Python

I just ran into this issue and had trouble filling in the "holes" left in the above example. Here's a more complete, working version that I used. Hopefully this helps someone:

# Custom formatter
class MyFormatter(logging.Formatter):

    err_fmt  = "ERROR: %(msg)s"
    dbg_fmt  = "DBG: %(module)s: %(lineno)d: %(msg)s"
    info_fmt = "%(msg)s"


    def __init__(self, fmt="%(levelno)s: %(msg)s"):
        logging.Formatter.__init__(self, fmt)

    
    def format(self, record):

        # Save the original format configured by the user
        # when the logger formatter was instantiated
        format_orig = self._fmt

        # Replace the original format with one customized by logging level
        if record.levelno == logging.DEBUG:
            self._fmt = MyFormatter.dbg_fmt

        elif record.levelno == logging.INFO:
            self._fmt = MyFormatter.info_fmt

        elif record.levelno == logging.ERROR:
            self._fmt = MyFormatter.err_fmt

        # Call the original formatter class to do the grunt work
        result = logging.Formatter.format(self, record)

        # Restore the original format configured by the user
        self._fmt = format_orig

        return result

Edit:

Compliments of Halloleo, here's an example of how to use the above in your script:

fmt = MyFormatter()
hdlr = logging.StreamHandler(sys.stdout)

hdlr.setFormatter(fmt)
logging.root.addHandler(hdlr)
logging.root.setLevel(DEBUG)

Edit 2:

Python3 logging has changed a bit. See here for a Python3 approach.

Solution 2 - Python

Yes, you can do this by having a custom Formatter class:

class MyFormatter(logging.Formatter):
    def format(self, record):
        #compute s according to record.levelno
        #for example, by setting self._fmt
        #according to the levelno, then calling
        #the superclass to do the actual formatting
        return s

Then attach a MyFormatter instance to your handlers.

Solution 3 - Python

And again like JS answer but more compact.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG :"DBG: %(module)s: %(lineno)d: %(message)s",
               logging.ERROR : "ERROR: %(message)s",
               logging.INFO : "%(message)s",
               'DEFAULT' : "%(levelname)s: %(message)s"}

    def format(self, record):
        self._fmt = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)
    
hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)

Solution 4 - Python

ONE WAY OF DOING THIS

Define a class

import logging

class CustomFormatter(logging.Formatter):
	"""Logging Formatter to add colors and count warning / errors"""

	grey = "\x1b[38;21m"
	yellow = "\x1b[33;21m"
	red = "\x1b[31;21m"
	bold_red = "\x1b[31;1m"
	reset = "\x1b[0m"
	format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s (%(filename)s:%(lineno)d)"

	FORMATS = {
		logging.DEBUG: grey + format + reset,
		logging.INFO: grey + format + reset,
		logging.WARNING: yellow + format + reset,
		logging.ERROR: red + format + reset,
		logging.CRITICAL: bold_red + format + reset
	}

	def format(self, record):
		log_fmt = self.FORMATS.get(record.levelno)
		formatter = logging.Formatter(log_fmt)
		return formatter.format(record)

Instantiate logger

# create logger with 'spam_application'
logger = logging.getLogger("My_app")
logger.setLevel(logging.DEBUG)

# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

ch.setFormatter(CustomFormatter())

logger.addHandler(ch)

And use!

logger.debug("debug message")
logger.info("info message")
logger.warning("warning message")
logger.error("error message")
logger.critical("critical message")

Result

Solution 5 - Python

Instead of relying on styles or internal fields, you could also create a Formatter that delegates to other formatters depending on record.levelno (or other criteria). This is a slightly cleaner solution in my humble opinion. Code below should work for any python version >= 2.7:

The simple way would look something like this:

class MyFormatter(logging.Formatter):

    default_fmt = logging.Formatter('%(levelname)s in %(name)s: %(message)s')
    info_fmt = logging.Formatter('%(message)s')

    def format(self, record):
        if record.levelno == logging.INFO:
            return self.info_fmt.format(record)
        else:
            return self.default_fmt.format(record)

But you could make it more generic:

class VarFormatter(logging.Formatter):

    default_formatter = logging.Formatter('%(levelname)s in %(name)s: %(message)s')

    def __init__(self, formats):
        """ formats is a dict { loglevel : logformat } """
        self.formatters = {}
        for loglevel in formats:
            self.formatters[loglevel] = logging.Formatter(formats[loglevel])

    def format(self, record):
        formatter = self.formatters.get(record.levelno, self.default_formatter)
        return formatter.format(record)

I used a dict as input here, but obviously you could also use tuples, **kwargs, whatever floats your boat. This would then be used like:

formatter = VarFormatter({logging.INFO: '[%(message)s]', 
                          logging.WARNING: 'warning: %(message)s'})
<... attach formatter to logger ...>

Solution 6 - Python

This is an adaptation of estani's answer to the new implementation of logging.Formatter which now relies on formatting styles. Mine relies on '{' style format, but it can be adapted. Could be refined to be more general and allow selection of formatting style and custom messages as arguments to __init__, too.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),
           logging.ERROR : logging._STYLES['{']("{module} ERROR: {message}"),
           logging.INFO : logging._STYLES['{']("{module}: {message}"),
           'DEFAULT' : logging._STYLES['{']("{module}: {message}")}

    def format(self, record):
        # Ugly. Should be better
        self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)

Solution 7 - Python

The above solution works with 3.3.3 release. However with 3.3.4 you get the following error.

FORMATS = { logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),

TypeError: 'tuple' object is not callable

After some searching around in the logging class Lib\logging_init_.py I found that a data structure has changed from 3.3.3 to 3.3.4 that causes the issue

3.3.3

_STYLES = {
    '%': PercentStyle,
    '{': StrFormatStyle,
    '$': StringTemplateStyle
}

3.3.4

_STYLES = {
   '%': (PercentStyle, BASIC_FORMAT),
   '{': (StrFormatStyle, '{levelname}:{name}:{message} AA'),
    '$': (StringTemplateStyle, '${levelname}:${name}:${message} BB'),
}

The updated solution is therefore

class SpecialFormatter(logging.Formatter):
     FORMATS = {logging.DEBUG : logging._STYLES['{'][0]("{module} DEBUG: {lineno}: {message}"),
       logging.ERROR : logging._STYLES['{'][0]("{module} ERROR: {message}"),
       logging.INFO : logging._STYLES['{'][0]("{module}: {message}"),
       'DEFAULT' : logging._STYLES['{'][0]("{module}: {message}")}

 def format(self, record):
    # Ugly. Should be better
    self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
    return logging.Formatter.format(self, record)

Solution 8 - Python

If you are just looking to skip formatting certain levels, you can do something simpler than the other answers like the following:

class FormatterNotFormattingInfo(logging.Formatter):
    def __init__(self, fmt = '%(levelname)s:%(message)s'):
        logging.Formatter.__init__(self, fmt)

    def format(self, record):
        if record.levelno == logging.INFO:
            return record.getMessage()
        return logging.Formatter.format(self, record)

This also has the advantage of working before and after the 3.2 release by not using internal variables like self._fmt nor self._style.

Content Type	Original Author	Original Content on Stackoverflow
Question	bedwyr	View Question on Stackoverflow
Solution 1 - Python	JS.	View Answer on Stackoverflow
Solution 2 - Python	Vinay Sajip	View Answer on Stackoverflow
Solution 3 - Python	estani	View Answer on Stackoverflow
Solution 4 - Python	Sergey Pleshakov	View Answer on Stackoverflow
Solution 5 - Python	Joris	View Answer on Stackoverflow
Solution 6 - Python	Evpok	View Answer on Stackoverflow
Solution 7 - Python	user1837990	View Answer on Stackoverflow
Solution 8 - Python	JDiMatteo	View Answer on Stackoverflow