_- Faster C++

I really enjoyed watching this talk about code performance on Youtube (Full video at the end). I thought it would be worth to share its content here. But the talk is so informative that a single blog post would never be enough to cover its entirety. So in this post I will solely focus on benchmarking. Precisely on how to evaluate code performance in a smarter way than what is usually done in the industry. The problem of comparing performance plots on various use cases is so frequent, yet so mishandled. It is surprising because interpreting timings, thus concluding on the best variation of a program, has a mathematical solution known since 1908! Waaaay before C++ was even designed! The solution is the Student test. It itself has a very interesting history, and I might write a future post about it! Simply put, the Student test provides a formal criterion to tell whether two sets of random samples belong to the same normal distribution. How to apply this test to benchmarking? Just run two variations of a program on a benchmark suite. This will give you two series of timings, most probably with different means and standard deviations. But think about it: Even if there was no variation at all in the program between the two runs, you would observe different means and standard deviations! This is due to the execution noise, which is why one does multiple runs of the same benchmark in the first place! So those differences must be marked as irrelevant, and discarded. This is why there is a need for a precise benchmarking criterion. You name it: the Student test. Given two sets of timings on two variations of a program, the Student test is able to tell whether those differences can be attributed to the execution noise, or one variation is superior to the other. Here is the full talk: