Code Matching Statistics -

Question

As I wrote in my previous topic: Benchmarking Code - Am I Doing It Right? I need to find a way to get control statistics, such as mean, standard deviation, etc. How can I do this using the methods I posted? Note that I am using the solution to test the code at a time interval, and not by calling a function many times. Any ideas?

I came up with only one, I don’t know if it is correct (pseudocode):

buffsize = 1024; buffer [buffsize]; totalcycles = 0 // arrays walltimeresults = [] cputimeresults = [] // benchmarking for i in (0, iterations): start = walltime(); fun2measure(args, buffer); end = walltime(); walltimeresults[i] = end - start; start = cputime(); fun2measure(args, buffer); end = cputime(); cputimeresults[i] = end - start; c1 = cyclecount(); fun2measure(args, buffer); c2 = cyclecount(); cyclesperbyte = c2-c1/(buffsize); totalcycles += cyclesperbyte; for i in range (0, iterations) : sum += walltimeresults[i]; avg_wall_time = sum / iterations; sum = 0; for i in range (0, iterations) : sum += cputimeresults[i]; avg_cpu_time = sum / iterations; avg_cycles = totalcycles / iterations; 

Is it correct? What about mean, standard deviation, etc.?

Answer 1

Your average value looks normal.

Mean value (i.e. average)

 mean = 1/N * sum( x[i] ) 

The standard deviation is the square root of the variance:

 sigma = sqrt( 1/N * sum( (x[i]-mean)^2 )