Qwen3.8-27B: faster than vLLM and SGLang on one GPU

Automated excerpt

Big models are where inference engines show off. Model: Qwen3.8-27B, BF16 weights with FP32 accumulation Baselines: vLLM 0.33 and SGLang, both with speculative decoding on Ahead of both engines at every one of the eight points. 8 workload points. 13 metrics each. 104 head-to-head comparisons per engine. vs vLLM 0.33: 104 of 104 SGLang is a very good engine. The full BF16 model verifies every single one.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.