Gemma 4 31B
Paddock vs vLLM vs SGLang vs llama.cpp · Text Generation · RTX PRO 6000 Blackwell Max-Q (96 GB) · 2026-07-20
Google's strongest Gemma 4 model - dense multimodal transformer with 5:1 sliding-window/global attention and a separate MTP drafter for speculative decoding.
Up to 1.8x
vs vLLM
Faster in 5 of 5 tests
Up to 2.2x
vs SGLang
Faster in 5 of 5 tests
Up to 9.4x
vs llama.cpp
Faster in 5 of 5 tests
We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.
Measured Throughput
Each engine ran the same tests, in tokens/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was. Every engine here uses speculative decoding: a small model guesses the next few words and the main model checks them in one go, so it produces several words at a time. Paddock does this by default, so the comparison is fair. These numbers rise and fall with the prompt and the load, and usually beat running without it. The dropdown shows the same engines with it off.
gemma-4-31B-it
Show the same engines without speculative decoding
Speculative decoding is off by default in these engines - it needs a separate helper model and extra setup. This is how they run out of the box. Paddock always uses it, so its bar is unchanged and shown for scale.
gemma-4-31B-it
Time to First Token
How long a client waits for the first token (median, best round), in milliseconds. Lower is better. The multiplier is how many times lower Paddock's latency is than that engine's on the same test.
| Test | Paddock | vLLM | SGLang | llama.cpp |
|---|---|---|---|---|
| 1 client · 1k-token prompts · 256 tokens out | 289 ms | 309 ms1.1x | 329 ms1.1x | 2,021 ms7.0x |
| 4 clients · 256-token prompts · 1k tokens out | 233 ms | 196 ms0.8x | 224 ms1.0x | 1,797 ms7.7x |
| 8 clients · 1k-token prompts · 256 tokens out | 597 ms | 379 ms0.6x | 455 ms0.8x | 4,305 ms7.2x |
| 8 clients · 4k-token prompts · 128 tokens out | 2,009 ms | 2,074 ms1.0x | 4,943 ms2.5x | 12,129 ms6.0x |
| 32 clients · 1k-token prompts · 256 tokens out | 1,451 ms | 4,357 ms3.0x | 5,788 ms4.0x | 12,100 ms8.3x |
Throughput Under Load
The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.
gemma-4-31B-it
Test Environment
Hardware
- GPU
- RTX PRO 6000 Blackwell Max-Q, 96 GB
- Driver
- 595.71.05
- CUDA
- 13.0
- OS
- Ubuntu 24.04, Linux 6.8.0-117
Engine Versions
- Paddock
- pre-release build
- vLLM
- 0.25.1
- vLLM (speculative)
- 0.25.1
- SGLang
- 0.5.15.post1
- SGLang (speculative)
- 0.5.15.post1
- llama.cpp
- b10068
- llama.cpp (speculative)
- b10068
Each engine ran the best setup it supports on this hardware, with everything on the GPU.
How We Tested
- Every engine ran on the same machine and GPU, never at the same time.
- The same test program sent identical requests to each engine.
- Every input was unique, so nothing could be served from a cache.
- Each number is the best timed round, after a warmup.
- We measure end to end: what an application connecting over HTTP actually gets.
- Where the engines differ in ways that could affect the comparison, we note it alongside the results.
Check our numbers
Paddock ships a benchmark harness, so you can run the same comparison on your own hardware.