Python comparison tool, for example, sockets?

Question

What I want

I would like to create a test suite for my Python project. I would like the performance of these tests to change when I introduced new code. I would like to do this in the same way that I test Python by running a utility command such as nosetests , and getting a beautifully formatted read.

What I like about nosetests

The nosetests tool works by looking at my directory structure for any functions called test_foo.py and runs all the test_bar() functions contained inside. It performs all these functions and prints whether they made an exception.

I would like something similar that looked for all bench_foo.py files and ran all the contained bench_bar() functions and reported their runtime.

Questions

Is there such a tool?

If not, what are some good starting points? Is the nose source suitable for this?

Answer 1

The wheezy documentation has a good example of how to do this with nose . The important part, if you just want to have timings, is to use the -q options to run -s , -s to not capture output (so you'll see the report output) and -m benchmark to run only 'time'.

I recommend using py.test for testing. To run an example from wheezy with this, change the name of the runTest method to test_bench_run and run only this test with:

 py.test -qs -k test_bench benchmark_hello.py 

( -q and -s , which have the same effect as the nose, and -k to select a test name pattern).

If you put your test tests in a file in a separate file or directory from ordinary tests, they are, of course, easier to select and do not need special names.

Answer 2

nosetests can run any type of test, so you can decide whether they test functionality, I / O validity, etc., performance or profiling (or whatever else you want). The Python Profiler is a great tool and it comes with your Python installation.

 import unittest import cProfile class ProfileTest(unittest.TestCase): test_run_profiler: cProfile.run('foo(bar)') cProfile.run('baz(bar)') 

You simply add a line to the test or add a test to the test case for all calls that you want to profile, and your main source is not polluted by the test code.

If you only want to run time and not all profiling information, timeit is another useful tool.