Is there a good test for compilers optimizing C ++?

Question

I evaluate the Visual C ++ 10 optimization compiler on trivial code samples to see how well the machine code comes out and there are no obvious creative possibilities yet.

Is there some kind of sample code base that is commonly used to evaluate how good an optimizing C ++ compiler is?

Answer 1

The only valid criterion is one that mimics the type of code you are developing. Optimizers react differently to different applications and different coding styles, and the only one that really counts is the code that you are going to compile using the compiler.

Answer 2

Try comparing libraries like Eigen (http://eigen.tuxfamily.org/index.php?title=Main_Page).

Answer 3

Several tests use scimark: http://math.nist.gov/scimark2/download_c.html , however you should be selective in what you test (check separately), as some tests may fail due to bad loop sweeps, but the rest of the code was excellent, but something even better only leads to a loop unwrap (i.e. the rest of the generated code was sub-parallel).

Answer 4

As already mentioned, you really need to measure optimization in the context of typical use cases for your own applications in typical target environments. For this reason, I include timers in my own set for automatic regression and found some unusual results described in the previous question . FWIW, I find VS2010 SP1, creating about 8% faster than VS2008 on my own application, about 13% when optimizing the entire program. It does not spread evenly in different cases. I also tend to see significant differences between lengthy test runs that are not visible, profiling much smaller test cases. I haven't done platform comparisons yet, for example. have many advantages for a platform or equipment.

I would suggest that many optimizers will be well tuned to give better results with respect to well-known reference sets, which in turn may mean that these are not the best code fragments that can be used to test the benefits of optimization. (Speculation, of course)