Why CUDA translation wont unlock AMD

Summary

Every few years, a new solution pops up promising the same dream: keep your CUDA codebase target AMD (and maybe other accelerators) no source rewrite no HIP porting “native performance” On paper, that sounds perfect. Take your existing CUDA applications, swap out the toolchain, and suddenly you’re “portable.” And to be fair: if you’re running research code or trying to get an internal tool to compile on a non-NVIDIA box, that can absolutely be useful. But if you care about actual performance on AMD, the kind that: reduces latency, wins benchmarks, squeezes every TFLOP from MI***-class accelerators, and doesn’t send people “back to NVIDIA” after one bad experiment, …then adopting a universal CUDA compatibility layer is the wrong long-term strategy. Not because the engineers behind these toolchains aren’t smart (they are), but because CUDA-first compilation will always be playing catch-up with what AMD exposes natively through ROCm, HIP, and vendor-tuned libraries. Why This Approach Is So Attractive These pitches are extremely compelling: “Develop your application using CUDA once and deploy it across various GPU platforms.” Concretely, these toolchains usually do something like this: Provide an nvcc-compatible compiler that accepts existing CUDA code, sometimes including inline PTX. Target AMD GPUs via LLVM backends instead of NVIDIA’s drivers. Implement the CUDA runtime, driver, and math APIs on top of AMD’s ROCm stack. Ship wrapper libraries that map CUDA-X APIs (e.g., cuBLAS/cuSOLVER) onto rocBLAS/rocSOLVER and friends. Maintain validation sets showing well-known CUDA projects compiling and running on AMD hardware. From a developer’s point of view, it feels magical: # On NVIDIA nvcc my_app.cu -o my_app_nvidia # On “everything” nvcc my_app.cu -o my_app_other_gpu For legacy CUDA-heavy HPC where a HIP/SYCL/ROCm rewrite would be painful, this is honestly a nice option. But that use case is very different from: “We want state-of-the-art LLM inference and training perfor...