We built our own inference engine and put it up against vLLM and SGLang on four models and three kinds of GPU.

Automated excerpt

Our engine, same model, same box: 3.24 seconds. Output throughput: 370.0 vs 74.9 tok/s (4.94×) Median TTFT: 3.24 s vs 357.5 s Per-request generation: 15.8 vs 7.7 tok/s (2.05×) We cleared work faster, so we kept taking more. vLLM never got past the first batch. Median TTFT hit 5.75× faster under steady traffic and 8.56× under burst.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.