Is there a way to remove stack traces in python log handlers using filters or some other mechanism when calling Logger.exception()?

Question

I know the subject sounds counter-intuitive but bear with me.

My goal is to have two rotating file log handlers, mostly for logger.exception() calls, one that includes stack traces and one that doesn't. I find that error conditions can sometimes cause the logs to fill with stack traces, making them difficult to read/parse without creating a separate parsing script, which I'd prefer not to do. I would like to use the built-in RotatingFileHandler class if possible.

Here's a simplified code snippet of what I'm trying to accomplish:

# getLogger just returns an instance of a logger derived from the main Python logger
#    where I can override the logger.exception() method

from myutils import getLogger

logger = getLogger(__name__)

try:
    x = 1 / 0
except ZeroDivisionError:
    # This one call should log to one file with the trace, one without the trace
    logger.exception("Oops")

I have the infrastructure in place to have this write to separate log files already using separate handlers, but they both include the stack traces. Is there a mechanism (logging filter or otherwise) where a log handler can strip the stack trace from logger.exception() calls?

I am assuming (or hoping) that logging filters attached to a handler can accomplish this, but I'm not sure how it can be done.

And just as an FYI, here is the source code of the Python logger for Logging.exception() and Logging.error() calls:

    def error(self, msg, *args, **kwargs):
        """
        Delegate an error call to the underlying logger.
        """
        self.log(ERROR, msg, *args, **kwargs)

    def exception(self, msg, *args, exc_info=True, **kwargs):
        """
        Delegate an exception call to the underlying logger.
        """
        self.log(ERROR, msg, *args, exc_info=exc_info, **kwargs)

vvvvv · Accepted Answer · 2025-03-22 10:07:53Z

1

You can use a custom Formatter that removes the exception info necessary to the logging of the stack trace. See the source code of the format method of the Formatter. You can see that since record.exc_info, record.exc_text and record.stack_info are set to None, no stack trace can be shown.

import logging

logger = logging.getLogger("foo")


class NoStackTraceFormatter(logging.Formatter):
    def format(self, record):
        record.exc_info = None
        record.exc_text = None
        record.stack_info = None
        return logging.Formatter.format(self, record)


handler_stacktrace = logging.FileHandler("stack.log")
handler_no_stacktrace = logging.FileHandler("nostack.log")
handler_no_stacktrace.setFormatter(NoStackTraceFormatter())

logger.addHandler(handler_stacktrace)
logger.addHandler(handler_no_stacktrace)

try:
    x = 1 / 0
except ZeroDivisionError:
    logger.exception("Oops")

Then check the 2 files:

# nostack.log
Oops

# stack.log
Oops
Traceback (most recent call last):
  File "main.py", line 23, in <module>
    x = 1 / 0
ZeroDivisionError: division by zero

edited Mar 22 at 10:07

answered Mar 21 at 21:39

vvvvv

32.9k19 gold badges70 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

DaveB Mar 22 at 0:26

This worked perfectly, thanks. One thing that I added was to specify the logging format to the NoStackTraceFormatter(), in my case "%(asctime)s\t%(levelname)s\t%(filename)s:%(lineno)s\t%(message)s".

DaveB · Accepted Answer · 2025-03-25 01:16:40Z

One update to the accepted answer from @vvvvv - the Python formatter caches the record object, so if one handler modifies it, the next handler will process the modified record rather than the original one. That caused some side effects depending on the order of the handlers. So, I passed in copies of the record object rather than the actual record to the formatter and that fixed the issue.

Here is what I've implemented. I also added a custom formatter that removed the stack trace messages but left one line showing any exception types that were raised.

class NoStackTraceFormatter(logging.Formatter):
    """Custom formatter to remove stack trace from log messages."""
    def format(self, record):
        """Removes all exception stack trace information from log messages."""
        temp_record = logging.LogRecord(record.name,
                                        record.levelno,
                                        record.pathname,
                                        record.lineno,
                                        record.msg,
                                        record.args,
                                        record.exc_info,
                                        record.funcName)
        temp_record.exc_info = None
        temp_record.exc_text = None
        temp_record.stack_info = None
        return logging.Formatter.format(self, temp_record)


class SimpleStackTraceFormatter(logging.Formatter):
    def format(self, record):
        """Remove the full stack trace from log messages but leave the lines
           in the stack trace that explicitly list the exception type raised.
        """
        temp_record = logging.LogRecord(record.name,
                                        record.levelno,
                                        record.pathname,
                                        record.lineno,
                                        record.msg,
                                        record.args,
                                        record.exc_info,
                                        record.funcName)
        if record.exc_info:
            # get rid of the stack trace except for lines that explicitly list the exception type raised
            if not record.exc_text:
                temp_record.exc_text = self.formatException(record.exc_info)
            if temp_record.exc_text:
                temp_record.exc_text = '\n'.join(
                    [f"    {line}" for line in temp_record.exc_text.splitlines() if '.exceptions.' in line]
                )
            temp_record.stack_info = None
        return logging.Formatter.format(self, temp_record)

An example of the output from the SimpleStackTraceFormatter is as follows:

2025-03-24 20:07:31,477 INFO    driver.py:439   Attempting to connect to the server
2025-03-24 20:07:34,513 ERROR   rest.py:921 Request timed out, will retry
    urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x000002989D18DC90>, 'Connection to x.x.x.x timed out. (connect timeout=3)')
    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='x.x.x.x', port=xxxx): Max retries exceeded with url: /path/ (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002989D18DC90>, 'Connection to x.x.x.x timed out. (connect timeout=3)'))
    requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='x.x.x.x', port=xxxx): Max retries exceeded with url: /path/ (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002989D18DC90>, 'Connection to x.x.x.x timed out. (connect timeout=3)'))

Collectives™ on Stack Overflow

Is there a way to remove stack traces in python log handlers using filters or some other mechanism when calling Logger.exception()?

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related