Three Fundamental Flaws of SIMD ISAs (2023)

Summary

According to Flynn’s taxonomy SIMD refers to a computer architecture that can process multiple data streams with a single instruction (i.e. “Single Instruction stream, Multiple Data streams”). There are different taxonomies, and within those several different sub-categories and architectures that classify as “SIMD”. In this post, however, I refer to packed SIMD ISA:s, i.e. the type of SIMD instruction set architecture that is most common in contemporary consumer grade CPU:s. More specifically, I refer to non-predicated packed SIMD ISA:s where the details of packed SIMD processing is exposed to the software environment. Packed SIMD The common trait of packed SIMD architectures is that several data elements are packed into a single register of a fixed width. Here is an example of possible configurations of a packed 128 bits wide SIMD register: For instance, a 128-bit register can hold sixteen integer bytes or four single precision floating-point values. This type of SIMD architecture has been wildly popular since the mid 1990s, and some packed SIMD ISA:s are: x86: MMX, 3DNow!, SSE, SSE2, …, and AVX, AVX2, AVX-5121ARM: ARMv6 SIMD, NEONPOWER: AltiVec (a.k.a. VMX and VelocityEngine)MIPS: MDMX, MIPS-3D, MSA, DSPSPARC: VISAlpha: MVI 1 AVX and later x86 SIMD ISA:s (especially AVX-512) incorporate features from vector processing, making them packed SIMD / vector processing hybrids (thus some of the aspects discussed in this article do not fully apply). The promise of all those ISA:s is increased data processing performance, since each instruction executes several operations in parallel. However, there are problems with this model. Flaw 1: Fixed register width Since the register size is fixed there is no way to scale the ISA to new levels of hardware parallelism without adding new instructions and registers. Case in point: MMX (64 bits) vs SSE (128 bits) vs AVX (256 bits) vs AVX-512 (512 bits). Adding new registers and instructions has many implications. For instance, the ABI...