VectorWare – from creators of `rust-GPU` and `rust-CUDA`

Summary

Our thesis Technology shifts happen gradually, then suddenly. We are in the suddenly part. New technologies like LLMs, generative AI, self-driving cars, drones, AR/VR, and robots are reshaping the world. But they are not the technology shift. They are the new applications enabled by it. The real shift is from CPU to GPU. The importance of CPUs and GPUs has inverted. To compete, CPUs are adding GPU features while GPUs are adding CPU features. CPUs and GPUs are converging. Software has not kept pace. CPU software is advanced, standardized, and familiar. GPU software is primitive, bespoke, and weird. Most programmers still focus on the CPU. We believe we are at the start of a new software industry. We intend to lead it. GPU-native software There are two broad classes of applications: GPU applications such as AI, computer vision, machine learning, scientific simulations, and graphics. These require GPUs and are driving most demand, investment, and improvements in compute hardware today. CPU applications, which includes nearly everything else. If you look at existing GPU applications, their software implementations aren't truly GPU-native. Instead, they are architected as traditional CPU software with a GPU add-on. For example, pytorch uses the CPU by default and GPU acceleration is opt-in. Even after opting in, the CPU is in control and orchestrates work on the GPU. Furthermore, if you look at the software kernels that run on the GPU they are simplistic with low cyclomatic complexity. This is not unique to pytorch. Most software is CPU-only, a small subset is GPU-aware, an even smaller subset is GPU-only, and no software is GPU-native. We are building software that is GPU-native. We intend to put the GPU in control. This does not happen today due to the difficulty of programming GPUs, the immaturity of GPU software and abstractions, and the relatively few developers targeting GPUs. With the advent of GPU databases, we are just starting to see CPU-based applications migr...