Show HN: Luminal – Open-source, search-based GPU compiler

Summary

Luminal is a deep learning library that uses search-based compilation to achieve high performance. ShowHN To run the demo shown on HN on mac, clone this repo and run: cd demos/matmul cargo run --release Important We're undergoing a large transition to "2.0", which introduces large-scale kernel search. This radically simplifies the compiler stack and allows us to discover complex optimizations entirely automatically. Please keep an eye on breaking changes, which usually are staged in the crates/luminal_2 before being merged into the main crate. Usage use luminal :: prelude :: * ; // Setup graph and tensors let mut cx = Graph :: new ( ) ; let a = cx . tensor ( ( 3 , 1 ) ) . set ( [ [ 1.0 ] , [ 2.0 ] , [ 3.0 ] ] ) ; let b = cx . tensor ( ( 1 , 4 ) ) . set ( [ [ 1.0 , 2.0 , 3.0 , 4.0 ] ] ) ; // Do math... let mut c = a . matmul ( b ) . retrieve ( ) ; // Compile and run graph cx . compile ( < ( GenericCompiler , CPUCompiler ) > :: default ( ) , & mut c ) ; cx . execute ( ) ; // Get result println ! ( "Result: {:?}" , c ) ; Getting Started Llama 3 8B the below is a quick example of how you can run Llama 3 8B locally using Luminal to go indepth on this example check out the documentation here cd ./examples/llama # Download the model bash ./setup/setup.sh # Run the model cargo run --release --features metal # MacOS (Recommended) cargo run --release --features cuda # Nvidia cargo run --release # CPU Features Speed Luminal can run Q8 Llama 3 8B on M-series Macbooks at 15-25 tokens per second. The goal is to become the fastest ML framework for any model on any device. Simplicity The core of luminal is and always will be minimal. It should be possible to understand the entire core library in an afternoon. RISC-style architecture Everything in luminal boils down to 12 primitive ops: Unary - Log2, Exp2, Sin, Sqrt, Recip Binary - Add, Mul, Mod, LessThan Other - SumReduce, MaxReduce, Contiguous These ops are enough to support transformers, convnets, etc. Speed We compile these ops ...