Stacktrace for userarning

Question

I see a warning similar to this in my logs:

py.warnings.__init__: WARNING .../bs4/__init__.py:219: UserWarning: "foo" looks like a filename, not markup. You should probably open this file and pass the filehandle into Beautiful Soup 

This post is not very helpful.

I would like to see stacktrace where this happens.

Please do not look at the contents of this warning. This question is not about perfect soup :-)

A simple solution is to change the third party code ( bs4/__init__.py on line 219) and add something like this:

 import traceback logger.warn('Exc at ...\n%s' % ''.join(traceback.format_stack())) 

But I would like to avoid this. Causes:

• This is a warning from the production system. I do not want to change the source.
• The next time a similar warning occurs, I would like to immediately see stacktrace

Is there a flag or parameter for python that I can change to see not only one line, but also stacktrace? I need the top frames to debug this.

This environment uses Python 2.7.

Answer 1

You will need to do the following:

• Create if USER_SITE does not exist: issue python -c "import site; site._script()" , see contents of USER_SITE variable

• Place the usercustomize.py file in this directory with the following code:

   import traceback import warnings _old_warn = warnings.warn def warn(*args, **kwargs): tb = traceback.extract_stack() _old_warn(*args, **kwargs) print("".join(traceback.format_list(tb)[:-1])) warnings.warn = warn 

  Credits to this answer for code.

Run the code as usual. My test code is:

 import warnings def f(): warnings.warn("foz") f() 

Before the procedure:

 $ python test_warn.py test_warn.py:4: UserWarning: foz warnings.warn("foz") 

After:

 $ python test_warn.py <USER_SITE_REDACTED>/usercustomize.py:6: UserWarning: foz _old_warn(*args, **kwargs) File "test_warn.py", line 6, in <module> f() File "test_warn.py", line 4, in f warnings.warn("foz") 

Answer 2

If I want to find the root of the warning, I usually push Warnings to Exceptions .

In your case, you can simply use warnings.simplefilter or warnings.filterwarnings .

For instance:

 import warnings def func(): warnings.warn('abc', UserWarning) def func2(): func() # Here I promote all UserWarnings to exceptions, but you could also use "warnings.filterwarnings" # to only promote warnings from a specified module or matching a specified message. # You may need to check which is most useful/appropriate for you. warnings.simplefilter("error", UserWarning) # you might need to reset this later :) func2() 

which gives a full trace:

 --------------------------------------------------------------------------- UserWarning Traceback (most recent call last) <ipython-input-11-be791e1071e7> in <module>() 8 9 warnings.simplefilter("error", UserWarning) ---> 10 func2() <ipython-input-11-be791e1071e7> in func2() 5 6 def func2(): ----> 7 func() 8 9 warnings.simplefilter("error", UserWarning) <ipython-input-11-be791e1071e7> in func() 2 3 def func(): ----> 4 warnings.warn('abc', UserWarning) 5 6 def func2(): UserWarning: abc 

And if you want to debug this, you can easily attach Python pdb to the last exception detected:

 import pdb pdb.pm() 

as a result of:

 > <ipython-input-11-be791e1071e7>(4)func() -> warnings.warn('abc', UserWarning) (Pdb) _________________ 

This begins a posthumous analysis of the last exception encountered. This should allow you to dig frames and check variables, etc.

---

You also asked about the flag, and there really is a flag that allows you to "warn processing", -W flag . This is very similar to the warnings.filterwarnings function. For convenience, I just copied the -W flag documentation here:

A warning. Pythons warning mechanism by default displays warning messages on sys.stderr. A typical warning message is as follows:

 file:line: category: message 

By default, each warning is printed once for each line of the source where it occurs. This setting determines how often alerts are printed.

You can specify several W parameters; when a warning matches several parameters, the action is taken for the last matching option. Invalid -W options are ignored (although the first time a warning is issued, a warning message about invalid parameters is displayed).

Starting with Python 2.7, DeprecationWarning and its descendants are ignored by default. The -Wd option can be used to re-enable.

Alerts can also be controlled from a Python program using the alert module.

The simplest form of argument is one of the following lines of action (or a unique abbreviation):

• ignore

  Ignore all warnings.

• Default

  Explicitly request the default behavior (print each warning once per source line).

• everything

  Print a warning every time it occurs (this can trigger many messages if the warning is repeated for the same source line, for example inside a loop).

• Module

  Print each warning only the first time it occurs in each module.

• once

  Print each warning only the first time it occurs in the program.

• Error:

  Raise an exception instead of printing a warning message.

The full form of the argument is:

 action:message:category:module:line 

Here, the action, as described above, applies only to messages that correspond to the remaining fields. Empty fields correspond to all values; trailing empty fields may be omitted. The message field corresponds to the beginning of the printed warning message; This match is not case sensitive. The category field corresponds to the category of alerts. It must be a class name; The match checks if the actual warning category is a subclass of the specified warning category. You must specify the fully qualified class name. The module field corresponds to the name (fully) of the module name; This match is case sensitive. The line field corresponds to the line number, where zero corresponds to all line numbers and, thus, is equivalent to the missing line number.