← All benchmarks

gpt-oss

Paddock vs vLLM vs SGLang vs llama.cpp · Text Generation · RTX PRO 6000 Blackwell Max-Q (96 GB) · 2026-07-20

OpenAI open-weight Mixture-of-Experts model (32 experts, 4 active/token) for lower latency and local or specialized use.

By OpenAIOpen model card ↗

RTX PRO 6000 Blackwell Max-Q · 2026-07-20

Up to 1.4x

vs vLLM

Faster in 4 of 5 tests

Up to 1.6x

vs SGLang

Faster in 5 of 5 tests

Up to 2.3x

vs llama.cpp

Faster in 5 of 5 tests

We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.

Measured Throughput

Each engine ran the same tests, in tokens/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was.

PaddockvLLMSGLangllama.cpp

gpt-oss-20b

vs vLLM: 1-1.4xvs SGLang: 1.1-1.6xvs llama.cpp: 1.2-2.3x
1 client · 1k-token prompts · 256 tokens out
Paddock
352.2 tokens/s
vLLM
252.31.4x
SGLang
223.81.57x
llama.cpp
291.61.21x
4 clients · 256-token prompts · 1k tokens out
Paddock
728.4 tokens/s
vLLM
677.11.08x
SGLang
625.01.17x
llama.cpp
612.01.19x
8 clients · 1k-token prompts · 256 tokens out
Paddock
1,144.6 tokens/s
vLLM
1,030.51.11x
SGLang
942.01.22x
llama.cpp
641.11.79x
8 clients · 4k-token prompts · 128 tokens out
Paddock
529.5 tokens/s
vLLM
527.21.0x
SGLang
465.51.14x
llama.cpp
292.81.81x
32 clients · 1k-token prompts · 256 tokens out
Paddock
2,456.2 tokens/s
vLLM
2,456.91.0x
SGLang
2,208.11.11x
llama.cpp
1,090.92.25x

Time to First Token

How long a client waits for the first token (median, best round), in milliseconds. Lower is better. The multiplier is how many times lower Paddock's latency is than that engine's on the same test.

TestPaddockvLLMSGLangllama.cpp
1 client · 1k-token prompts · 256 tokens out43 ms56 ms1.3x47 ms1.1x117 ms2.7x
4 clients · 256-token prompts · 1k tokens out65 ms60 ms0.9x46 ms0.7x197 ms3.0x
8 clients · 1k-token prompts · 256 tokens out268 ms252 ms0.9x316 ms1.2x643 ms2.4x
8 clients · 4k-token prompts · 128 tokens out1,108 ms1,064 ms1.0x918 ms0.8x1,659 ms1.5x
32 clients · 1k-token prompts · 256 tokens out908 ms872 ms1.0x801 ms0.9x1,095 ms1.2x

Throughput Under Load

The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.

gpt-oss-20b

PaddockvLLMSGLangllama.cpp
tokens/s06251,2501,8752,5001 client1k in · 256 out4 clients256 in · 1k out8 clients1k in · 256 out8 clients4k in · 128 out32 clients1k in · 256 outvLLM - 1 client · 1k-token prompts · 256 tokens out: 252.3 tokens/sSGLang - 1 client · 1k-token prompts · 256 tokens out: 223.8 tokens/sllama.cpp - 1 client · 1k-token prompts · 256 tokens out: 291.6 tokens/sPaddock - 1 client · 1k-token prompts · 256 tokens out: 352.2 tokens/s1.2×vLLM - 4 clients · 256-token prompts · 1k tokens out: 677.1 tokens/sSGLang - 4 clients · 256-token prompts · 1k tokens out: 625.0 tokens/sllama.cpp - 4 clients · 256-token prompts · 1k tokens out: 612.0 tokens/sPaddock - 4 clients · 256-token prompts · 1k tokens out: 728.4 tokens/s1.1×vLLM - 8 clients · 1k-token prompts · 256 tokens out: 1,030.5 tokens/sSGLang - 8 clients · 1k-token prompts · 256 tokens out: 942.0 tokens/sllama.cpp - 8 clients · 1k-token prompts · 256 tokens out: 641.1 tokens/sPaddock - 8 clients · 1k-token prompts · 256 tokens out: 1,144.6 tokens/s1.1×vLLM - 8 clients · 4k-token prompts · 128 tokens out: 527.2 tokens/sSGLang - 8 clients · 4k-token prompts · 128 tokens out: 465.5 tokens/sllama.cpp - 8 clients · 4k-token prompts · 128 tokens out: 292.8 tokens/sPaddock - 8 clients · 4k-token prompts · 128 tokens out: 529.5 tokens/s1.0×vLLM - 32 clients · 1k-token prompts · 256 tokens out: 2,456.9 tokens/sSGLang - 32 clients · 1k-token prompts · 256 tokens out: 2,208.1 tokens/sllama.cpp - 32 clients · 1k-token prompts · 256 tokens out: 1,090.9 tokens/sPaddock - 32 clients · 1k-token prompts · 256 tokens out: 2,456.2 tokens/s1.0×+114 tokens/s

Test Environment

Hardware

GPU
RTX PRO 6000 Blackwell Max-Q, 96 GB
Driver
595.71.05
CUDA
13.0
OS
Ubuntu 24.04, Linux 6.8.0-117

Engine Versions

Paddock
pre-release build
vLLM
0.25.1
SGLang
0.5.15.post1
llama.cpp
b10069

Each engine ran the best setup it supports on this hardware, with everything on the GPU.

RTX 5090 · 2026-07-20

Up to 1.4x

vs vLLM

Faster in 5 of 6 tests

Up to 2.7x

vs SGLang

Faster in 5 of 6 tests

Up to 3.2x

vs llama.cpp

Faster in 6 of 6 tests

We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.

Measured Throughput

Each engine ran the same tests, in tokens/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was.

PaddockvLLMSGLangllama.cpp

gpt-oss-20b

vs vLLM: 0.7-1.4xvs SGLang: 0.7-2.7xvs llama.cpp: 1.3-3.2x
1 client · 1k-token prompts · 256 tokens out
Paddock
386.2 tokens/s
vLLM
276.51.4x
SGLang
246.01.57x
llama.cpp
308.71.25x
4 clients · 256-token prompts · 1k tokens out
Paddock
862.7 tokens/s
vLLM
729.51.18x
SGLang
662.51.3x
llama.cpp
631.51.37x
8 clients · 1k-token prompts · 256 tokens out
Paddock
1,361.7 tokens/s
vLLM
1,106.31.23x
SGLang
1,012.21.35x
llama.cpp
657.62.07x
8 clients · 4k-token prompts · 128 tokens out
Paddock
677.5 tokens/s
vLLM
547.21.24x
SGLang
492.51.38x
llama.cpp
348.61.94x
16 clients · agentic-token prompts · 256 tokens out
Paddock
1,431.3 tokens/s
vLLM
1,978.00.72x
SGLang
1,941.80.74x
llama.cpp
773.21.85x
32 clients · 1k-token prompts · 256 tokens out
Paddock
3,047.5 tokens/s
vLLM
2,723.41.12x
SGLang
1,132.22.69x
llama.cpp
948.03.21x

Time to First Token

How long a client waits for the first token (median, best round), in milliseconds. Lower is better. The multiplier is how many times lower Paddock's latency is than that engine's on the same test.

TestPaddockvLLMSGLangllama.cpp
1 client · 1k-token prompts · 256 tokens out36 ms57 ms1.6x47 ms1.3x116 ms3.2x
4 clients · 256-token prompts · 1k tokens out52 ms60 ms1.1x49 ms0.9x218 ms4.2x
8 clients · 1k-token prompts · 256 tokens out204 ms255 ms1.2x220 ms1.1x637 ms3.1x
8 clients · 4k-token prompts · 128 tokens out841 ms577 ms0.7x793 ms0.9x1,142 ms1.4x
16 clients · agentic-token prompts · 256 tokens out979 ms106 ms0.1x63 ms0.1x514 ms0.5x
32 clients · 1k-token prompts · 256 tokens out773 ms475 ms0.6x604 ms0.8x1,163 ms1.5x

Throughput Under Load

The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.

gpt-oss-20b

PaddockvLLMSGLangllama.cpp
tokens/s01,0002,0003,0004,0001 client1k in · 256 out4 clients256 in · 1k out8 clients1k in · 256 out8 clients4k in · 128 out16 clientsagentic in · 256 out32 clients1k in · 256 outvLLM - 1 client · 1k-token prompts · 256 tokens out: 276.5 tokens/sSGLang - 1 client · 1k-token prompts · 256 tokens out: 246.0 tokens/sllama.cpp - 1 client · 1k-token prompts · 256 tokens out: 308.7 tokens/sPaddock - 1 client · 1k-token prompts · 256 tokens out: 386.2 tokens/s1.3×vLLM - 4 clients · 256-token prompts · 1k tokens out: 729.5 tokens/sSGLang - 4 clients · 256-token prompts · 1k tokens out: 662.5 tokens/sllama.cpp - 4 clients · 256-token prompts · 1k tokens out: 631.5 tokens/sPaddock - 4 clients · 256-token prompts · 1k tokens out: 862.7 tokens/s1.2×vLLM - 8 clients · 1k-token prompts · 256 tokens out: 1,106.3 tokens/sSGLang - 8 clients · 1k-token prompts · 256 tokens out: 1,012.2 tokens/sllama.cpp - 8 clients · 1k-token prompts · 256 tokens out: 657.6 tokens/sPaddock - 8 clients · 1k-token prompts · 256 tokens out: 1,361.7 tokens/s1.2×vLLM - 8 clients · 4k-token prompts · 128 tokens out: 547.2 tokens/sSGLang - 8 clients · 4k-token prompts · 128 tokens out: 492.5 tokens/sllama.cpp - 8 clients · 4k-token prompts · 128 tokens out: 348.6 tokens/sPaddock - 8 clients · 4k-token prompts · 128 tokens out: 677.5 tokens/s1.2×vLLM - 16 clients · agentic-token prompts · 256 tokens out: 1,978.0 tokens/sSGLang - 16 clients · agentic-token prompts · 256 tokens out: 1,941.8 tokens/sllama.cpp - 16 clients · agentic-token prompts · 256 tokens out: 773.2 tokens/sPaddock - 16 clients · agentic-token prompts · 256 tokens out: 1,431.3 tokens/s0.7×vLLM - 32 clients · 1k-token prompts · 256 tokens out: 2,723.4 tokens/sSGLang - 32 clients · 1k-token prompts · 256 tokens out: 1,132.2 tokens/sllama.cpp - 32 clients · 1k-token prompts · 256 tokens out: 948.0 tokens/sPaddock - 32 clients · 1k-token prompts · 256 tokens out: 3,047.5 tokens/s1.1×+324 tokens/s

Test Environment

Hardware

GPU
RTX 5090, 32 GB

Engine Versions

Paddock
pre-release build
vLLM
0.25.1 (latest PyPI; CUDA 13 wheels, MARLIN mxfp4 MoE + FlashInfer sampling auto-selected)
SGLang
0.5.15.post1 (latest PyPI; installed with --prerelease=allow for flash-attn-4==4.0.0b15; the cutlass-dsl GPUModuleOp startup crash from the runbook is fixed in this post-release)
llama.cpp
latest master @ 178a6c44, ~b10069, committed 2026-07-19 (built on-box, CUDA sm_120, GNU 13.3)

Each engine ran the best setup it supports on this hardware, with everything on the GPU.

How We Tested

  • Every engine ran on the same machine and GPU, never at the same time.
  • The same test program sent identical requests to each engine.
  • Every input was unique, so nothing could be served from a cache.
  • Each number is the best timed round, after a warmup.
  • We measure end to end: what an application connecting over HTTP actually gets.
  • Where the engines differ in ways that could affect the comparison, we note it alongside the results.

Check our numbers

Paddock ships a benchmark harness, so you can run the same comparison on your own hardware.

About PaddockGet Early Access