How to find out where a Python Warning is from

PythonDebuggingWarningsPandas

Python Problem Overview


I'm still kinda new with Python, using Pandas, and I've got some issues debugging my Python script.

I've got the following warning message :

[...]\pandas\core\index.py:756: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
return self._engine.get_loc(key)

And can't find where it's from.

After some research, I tried to do that in the Pandas lib file (index.py):

try:
    return self._engine.get_loc(key)
except UnicodeWarning:
    warnings.warn('Oh Non', stacklevel=2)

But that didn't change anything about the warning message.

Python Solutions


Solution 1 - Python

You can filter the warnings to raise which will enable you to debug (e.g. using pdb):

import warnings
warnings.filterwarnings('error')

*The warnings filter can be managed more finely (which is probably more appropriate) e.g.:

warnings.filterwarnings('error', category=UnicodeWarning)
warnings.filterwarnings('error', message='*equal comparison failed*')

Multiple filters will be looked up sequentially. ("Entries closer to the front of the list override entries later in the list, if both match a particular warning.")

Solution 2 - Python

You can also use the commandline to control the warnings:

python -W error::UnicodeWarning your_code.py

From the man page:

> -W argument
> [...] error to raise an exception instead of printing a warning message.

This will have the same effect as putting the following in your code:

import warnings
warnings.filterwarnings('error', category=UnicodeWarning)

As was already said in Andy's answer.

Solution 3 - Python

If you enable logging in python, then when an exception is received you can use the method logging.exception to log when an exception has been caught - this method will print out a nicely formatted stack trace that shows you exactly in the code where the exception originated. See the python document on logging for more information.

import logging
log = logging.getLogger('my.module.logger')

try:
    return self._engine.get_loc(key)
except UnicodeWarning:
    log.exception('A log message of your choosing')

Alternatively, you can get a tuple that contains details of the exception in your code by calling sys.exc_info() (this requires you to import the sys module).

Solution 4 - Python

The most informative way to investigate a warning is to convert it into an error (Exception) so you can see its full stacktrace:

import warnings
warnings.simplefilter("error")

See warnings.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionBonswouarView Question on Stackoverflow
Solution 1 - PythonAndy HaydenView Answer on Stackoverflow
Solution 2 - PythonMaxime R.View Answer on Stackoverflow
Solution 3 - PythonrobjohncoxView Answer on Stackoverflow
Solution 4 - PythonArthurView Answer on Stackoverflow