The messy reality of SIMD (vector) functions

Summary

We’ve discussed SIMD and vectorization extensively on this blog, and it was only a matter of time before SIMD (or vector) functions came up. In this post, we explore what SIMD functions are, when they are useful, and how to declare and use them effectively.A SIMD function is a function that processes more than one piece of data. Take for example a mathematical sin function:double sin(double angle);This function takes one double and returns one double. The vector version that processes four values in a single function would look like this:double[4] sin(double angle[4]);Or, we were to use native AVX SIMD types, it could look like this:__m256d sin(__m256 d);The basic idea behind vector functions is to improve performance by processing multiple data elements per function call, either through manually vectorized functions or compiler-generated vector functions.Why do we even need vector functions?The term vector function can refer to two related concepts:A function that takes or returns vector types directly (e.g, __m256d sin(__m256d)).An implementation of C/C++ functions which receives and/or returns vector types, and which the compiler can use during auto-vectorization (e.g. double sin(double) can have several implementation. One implementation is a scalar, another can work with two doubles double[2] sin(double[2]), another with four doubles double[4] sin(double[4]). In an effort to vectorize a certain loop, the compiler will chose among one of those implementations.)In this post we will talk about the second type of vector functions, i.e. vector functions that the compiler can pick to automatically vectorize loops. So, a compiler could take a following loop:for (size_t i = 0; i < n; i++) { res[i] = sin(in[i]); }And call either the scalar version of sin or a vector version of sin – typically it will opt for vector version for performance reasons, but might also call the scalar version to deal with few iterations of the loop that cannot be processed in vector code.Do yo...