Python: counting how many times a given string is executed

Question

Problem

For educational purposes, I would like to calculate how many times a given line is executed in a given function without modifying or decorating it . For example, for a function:

def binary_search(seq, x): (a, b) = (0, len(seq) - 1) while a <= b: m = (a + b) / 2 if x < seq[m]: b = m - 1 elif x > seq[m]: a = m + 1 else: return m 

I would just write something like this:

 print count_exec(binary_search, range(100), 44, line_number = 4) 

... or even so:

 print count_exec(binary_search(range(100), 44), line = "m = (a + b) / 2") 

... which both should print the number of times when the 4th line is executed (which equals 7). The ultimate goal is to provide an empirical approach to the complexity of any function:

Not decisions

My current solution is to add a function attribute:

 def binary_search(seq, x): binary_search.count = 0 # <------------------ added (a, b) = (0, len(seq) - 1) while a <= b: binary_search.count += 1 # <------------- added m = (a + b) / 2 if x < seq[m]: b = m - 1 elif x > seq[m]: a = m + 1 else: return m binary_search(range(100), 44) print binary_search.count 

I think I could create a decorated count_this_line function:

 def binary_search(seq, x): (a, b) = (0, len(seq) - 1) while a <= b: count_this_line() # <-------------------- added m = (a + b) / 2 ... 

Perhaps you can decorate the binary_search function, but for me it is considered a modification of it.

Ideas

• The ast standard library can get an abstract syntax tree of any given script and even execute it.
• I have little experience using the Python profiler. I think it is very difficult for my needs. Could this be the way?

Answer 1

You can use the line_profiler module for this ( see docs ). Note that I needed to get a compatible version 3.x from a forked repo - not sure if it hasn't merged yet.

For instance. I put your binary search function in a file and then added:

 prof = profile(binary_search) prof(range(100), 44) 

This is the same as the @profile decorator as stated in the docs, but you don't need to change the source code. I ran

 kernprof.py -l binsearch.py python -m line_profiler binsearch.py.lprof 

And this popped up:

 Function: binary_search at line 1 Total time: 4.1e-05 s Line # Hits Time Per Hit % Time Line Contents ============================================================== 1 def binary_search(seq, x): 2 1 6 6.0 14.6 (a, b) = (0, len(seq) - 1) 3 7 8 1.1 19.5 while a <= b: 4 7 7 1.0 17.1 m = (a + b) // 2 5 7 8 1.1 19.5 if x < seq[m]: 6 2 2 1.0 4.9 b = m - 1 7 5 5 1.0 12.2 elif x > seq[m]: 8 4 4 1.0 9.8 a = m + 1 9 else: 10 1 1 1.0 2.4 return m 

"Hits" is the number you are looking for. As a bonus, you also get time information, although that would be more accurate with many performances.

Answer 2

Following Jason's suggestion, I wrote a clean Python solution:

 import line_profiler import __builtin__ import cStringIO import re def profile(path, function_call, line_number): prof = line_profiler.LineProfiler() __builtin__.__dict__['profile'] = prof script = open(path).read() ns = locals() function_name = function_call[:function_call.index("(")] rex = re.compile("((?ms)^def %s.+)" % function_name) script = rex.sub(r"@profile\n\1\n%s" % function_call, script) exec(script, ns, ns) stream = cStringIO.StringIO() prof.print_stats(stream) s = stream.getvalue() stream.close() return int(re.search(r"(?m)^\s*%s\s*(\S*)" % (line_number+1), s).group(1)) if __name__ == "__main__": print profile("binary_search.py", "binary_search(range(100), 44)", 3) 

It reads the source of the script containing the function for profiling, decorates it, adds the desired call to the end, executes it, outputs statistics to the string, extracts the number of hits at the specified line number, and returns it as an int . It works as needed, but with a significant decrease in performance.

Perhaps the best solution would be to drop the profiler, but at the same time keep the idea of ​​decorating and executing the source code on the fly. I will edit my answer if I implant.

Anyway, thanks to Jason for giving me a way out!