Python warning log stack trace

Question

The package that I use in my python program throws a warning that I would like to understand the exact reason. I set logging.captureWarning(True) and I am fixing a warning in my protocol, but still do not know where it logging.captureWarning(True) from. How do I also log a stack trace so that I can see where the warning appears in my code? Do I traceback ?

Answer 1

I ended up working with below:

 import warnings import traceback _formatwarning = warnings.formatwarning def formatwarning_tb(*args, **kwargs): s = _formatwarning(*args, **kwargs) tb = traceback.format_stack() s += ''.join(tb[:-1]) return s warnings.formatwarning = formatwarning_tb logging.captureWarnings(True) 

Answer 2

This is a bit hacky, but you can defuse the warnings.warn method as follows:

 import traceback import warnings def g(): warnings.warn("foo", Warning) def f(): g() warnings.warn("bar", Warning) _old_warn = warnings.warn def warn(*args, **kwargs): tb = traceback.extract_stack() _old_warn(*args, **kwargs) print("".join(traceback.format_list(tb)[:-1])) warnings.warn = warn f() print("DONE") 

This is the conclusion:

 /tmp/test.py:14: Warning: foo _old_warn(*args, **kwargs) File "/tmp/test.py", line 17, in <module> f() File "/tmp/test.py", line 8, in f g() File "/tmp/test.py", line 5, in g warnings.warn("foo", Warning) /tmp/test.py:14: Warning: bar _old_warn(*args, **kwargs) File "/tmp/test.py", line 17, in <module> f() File "/tmp/test.py", line 9, in f warnings.warn("bar", Warning) DONE 

See that calling the original warnings.warn function warnings.warn not report the line you want, the trace pin is really correct (you can print the warning yourself).

Answer 3

If you don't know what data / instructions are causing the warning, you can use tools like the standard Python Debugger .

The documentation is really good and detailed, but some examples that might help should be:

• Without changing the source code: calling debbugger as a script:

  $ python -m pdb myscript.py

• Changing the source code: you can use pdb.set_trace() calls, which work as breakpoints; For example, consider that I have the following example :

   x = 2 x = x * 10 * 100 y = x + 3 + y return y 

  And I would like to know what value x and y have before returning or what the stack contains, I would add the following line between these statements:

  pdb.set_trace()

  And I will be prompted (Pdb) to let you go through the code line by line. Useful Prompt Commands (Pdb) :

  • n : executes the following statement.
  • q : terminates the entire program.
  • c : issues a prompt (Pdb) and stops debugging.
  • p varname: prints the value of varname

Since you are not providing more information, I do not know if this is enough, but I think that at least this could be a good start.

BONUS EDIT

Based on this answer , I found a good and convenient GUI debugging tool that you can simply install:

$ pip install pudb

And run the debugger using a script with

$ python -m pudb.run myscript.py

EDIT: adding postmortem debugging

If we don’t even know if the code will crash or not, we can enter postmortem debugging if a crash occurs. From the Pbd documentation:

Typical use for checking a broken program:

 >>> import pdb >>> import mymodule >>> mymodule.test() Traceback (most recent call last): File "<stdin>", line 1, in ? File "./mymodule.py", line 4, in test test2() File "./mymodule.py", line 3, in test2 print spam NameError: spam >>> pdb.pm() > ./mymodule.py(3)test2() -> print spam (Pdb) 

Since postmortem looks at sys.last_traceback , enter only if there is a trace (and so on, a warning or failure):

 if sys.last_traceback: pdb.pm() 

Answer 4

You can include warnings in exceptions, which means that you will automatically get a stack trace:

 warnings.filterwarnings("error") 

See https://docs.python.org/3.4/library/warnings.html#the-warnings-filter

Answer 5

If it were me, I would go with @ Lluís Vilanova a quick and dirty hack, just to find something. But if this is not an option ...

If you really want a "logging" solution, you can try something like this (fully working source).

The main steps:

• Create your own logging.Formatter subdirectory that includes the current stack in which the log entry is written
• Use this formatter in the warning class.

Soft code is a custom formatter:

 class Formatter(logging.Formatter): def format(self, record): record.stack_info = ''.join(traceback.format_stack()) return super().format(record) 

In docs :

 New in version 3.2: The stack_info parameter was added.