NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

Summary

Thanks to NVIDIA’s early access program, we are thrilled to get our hands on the NVIDIA DGX™ Spark. It’s quite an unconventional system, as NVIDIA rarely releases compact, all-in-one machines that bring supercomputing-class performance to a desktop workstation form factor. Over the past year, SGLang has been rapidly expanding its developer base in the datacenter segment, recognized by the inference community for its great performance. Successfully deploying DeepSeek with Prefill-decode Disaggregation (PD) and Expert Parallelism (EP) at large scale, running on both 96 NVIDIA H100 GPU clusters and the latest GB200 NVL72 systems, SGLang has continually pushed the boundaries of large-scale inference performance and developer productivity. Inspired by the capabilities of the DGX Spark, for the first time, SGLang is now expanding beyond the datacenter and into the consumer market, bringing its proven inference framework directly to developers and researchers everywhere. In this review, we’ll be taking a close look at this beautiful machine, from its exterior aesthetics to its performance and use cases. Also check out our video review here. Exterior The DGX Spark is a gorgeous piece of engineering. It features a full-metal chassis with a sleek champagne-gold finish. Both the front and rear panels are built with metal foam, reminding me of the design of NVIDIA DGX A100 and H100. Around the back, the DGX Spark offers an impressive array of connectivity options: a power button, four USB-C ports (with the leftmost supporting up to 240 W of power delivery), an HDMI port, a 10 GbE RJ-45 Ethernet port, and two QSFP ports driven by NVIDIA ConnectX-7 NIC capable of up to 200 Gbps. These interfaces allow two DGX Spark units to be connected together, allowing them to run even larger AI models. The use of USB Type-C for power delivery is a particularly interesting design choice, one that’s virtually unheard of on other desktop machines. Comparable systems like the Mac Mini or Mac Stud...