AI
vLLM reports near-parity with DeepSeek’s engine and major rack-scale gains
The open-source vLLM serving engine reports decode performance within 6% of DeepSeek’s specialized DeepInfer runtime on DeepSeek-V4.1-Flash, alongside a 7.9-fold gain over a single-node baseline on a 72-GPU GB300 rack. The results come from combining optimized kernels with prefill-decode separation, wide expert parallelism and load balancing. Reported improvements include up to 44% higher prefill throughput, but production gains will depend on model architecture, traffic, networking and CPU capacity.
Editorial summary · Publisher article
What matters
- The benchmark reports 954,000 tokens per second per GPU on an NVL72 rack.
- Expert placement improved a balance score from 0.35 to 0.94 for the target model.
- Host-side scheduling became a bottleneck after GPU execution was optimized.
Context & caveats
- The results are published benchmarks, not independent production validation.
Why this made the edition
A technically detailed engineering report gives benchmark conditions, mechanisms, measured gains and explicit limits on generalization.
Source material