AI article
I made a language model that lives in your CPU's L3 cache
Community description: 3,405 tok/s on one core of a $60 CPU: a 1.9 MB model trained from scratch and sized to fit inside L3 cache — code and measurements on GitHub.
Dev.to | Oct 12, 2026 | Mohammad Nazibul Islam
Automated excerpt
Every forward pass of a language model reads its weights once. My model: 1. 91M parameters, int8 → 1. 97 MB of weights 1. 97 / 4 = 49. 8% of L3 If the weights fit in cache, decode is a cache-streaming problem: ~6. 7 GB/s of sequential reads served by L3. The numbers are from a Ryzen 5 3400G (4 MB L3).
Selected automatically from source text; not independently written or fact-checked. Read the original for full context.