afterburn·For inference teams
Compare engines
on your own traffic.
Run vLLM, SGLang and TensorRT-LLM side by side on the same model and the same prompts, keep every run, and move traffic only when the numbers say so.
- Same prompts, every engine
- Runs kept as baselines
- Engine metrics beside latency
The work
Tuning without
a spreadsheet.
- Engine against engine
One benchmark, several endpoints serving the same model: first token, between tokens, total time and errors, at the concurrency you run.
- Regressions caught
Rerun last week’s benchmark after an upgrade and read the two runs side by side before any traffic moves.
- Safe moves
Change a route’s primary, keep the previous one as fallback, and roll back in one step if the after looks worse than the before.
A comparison
Same model,
three engines.
One run, laid out the way the console shows it.
Run b-0142 · first token p95Sample data
- vllm-h100
- 175 ms
- sglang-l40s
- 232 ms
- trtllm-h100
- 151 ms
- limit
- 300 ms
Run b-0142 · cost per 1M outputSample data
- vllm-h100
- $0.21
- sglang-l40s
- $0.17
- trtllm-h100
- $0.24
- selected
- sglang-l40s, cheapest within the limit
Measure the
next upgrade.
Keep a baseline now, and the next engine release has something to beat.