AI article
My Triton Kernel Ties bitsandbytes — and Both Lose to Math
Community description: A fused 4-bit Triton GEMV for batch-1 decoding, written after computing the memory-bandwidth ceiling. Ties bitsandbytes, 16x under the roof.
Dev.to | Oct 12, 2026 | Raihan
Automated excerpt
A fused W4A16 GEMV for batch-1 decoding, measured against the RTX 3060 roofline. Batch-1 decoding is memory-bandwidth bound: each token reads every weight byte once. The int4 ceiling (412 tok/s) sits 16× above measured.
Selected automatically from source text; not independently written or fact-checked. Read the original for full context.