SGLang vs vLLM: Your Agent Is Slow Because You Picked the Wrong Cache

Automated excerpt

Both vLLM's prefix caching and SGLang's RadixAttention reuse previously computed KV cache for repeated prompt prefixes. The second Dev. to comparison (ifnodoraemon's SGLang vs vLLM architecture showdown) makes the mechanism visible in the metrics: at a 4,096-token prompt with 75% prefix overlap and concurrency 64, SGLang hit a 78. 6% KV cache hit rate vs vLLM's 41. 2%, with median TTFT of 85ms vs 380ms (4. 47x), P99 TTFT of 280ms vs 1,250ms, and throughput of 5,430 vs 3,120 tok/s. SGLang, large margin (11. 57x vs 5. 21x TTFT speedup) SGLang (4.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.