← All benchmarks

Qwen3 Reranker

Paddock vs vLLM vs SGLang · Reranking · RTX 5090 (32 GB) · 2026-07-20

Qwen3 reranker scoring query-document relevance across languages and domains (relevance-score output).

By AlibabaOpen model card ↗

RTX 5090 · 2026-07-20

Up to 2.4x

vs vLLM

Faster in 8 of 8 tests

Up to 2.9x

vs SGLang

Faster in 11 of 11 tests

We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.

Measured Throughput

Each engine ran the same tests, in docs/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was.

PaddockvLLMSGLang

Qwen3-Reranker-0.6B

vs vLLM: 1.6-2.4xvs SGLang: 2.2-2.9x
1 client · 32 docs per request
Paddock
2,465.2 docs/s
vLLM
1,006.32.45x
SGLang
857.02.88x
1 client · 128 docs per request
Paddock
2,993.8 docs/s
vLLM
1,265.72.37x
SGLang
1,304.12.3x
4 clients · 32 docs per request
Paddock
3,158.6 docs/s
vLLM
1,957.81.61x
SGLang
1,370.42.3x
16 clients · 8 docs per request
Paddock
2,879.8 docs/s
vLLM
1,834.51.57x
SGLang
1,310.22.2x

Qwen3-Reranker-4B

vs vLLM: 1.4-1.8xvs SGLang: 1.4-1.8x
1 client · 32 docs per request
Paddock
583.7 docs/s
vLLM
320.91.82x
SGLang
322.91.81x
1 client · 128 docs per request
Paddock
610.6 docs/s
vLLM
363.11.68x
SGLang
427.11.43x
4 clients · 32 docs per request
Paddock
617.8 docs/s
vLLM
426.91.45x
SGLang
413.01.5x
16 clients · 8 docs per request
Paddock
582.4 docs/s
vLLM
428.31.36x
SGLang
417.61.39x

Qwen3-Reranker-8B

vs SGLang: 1.4-1.5x
1 client · 32 docs per request
Paddock
326.2 docs/s
SGLang
213.71.53x
4 clients · 32 docs per request
Paddock
340.5 docs/s
SGLang
244.01.4x
16 clients · 8 docs per request
Paddock
329.1 docs/s
SGLang
240.11.37x

Throughput Under Load

The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.

Qwen3-Reranker-0.6B

PaddockvLLMSGLang
docs/s01,0002,0003,0004,0001 client32 per request1 client128 per request4 clients32 per request16 clients8 per requestvLLM - 1 client · 32 docs per request: 1,006.3 docs/sSGLang - 1 client · 32 docs per request: 857.0 docs/sPaddock - 1 client · 32 docs per request: 2,465.2 docs/s2.4×vLLM - 1 client · 128 docs per request: 1,265.7 docs/sSGLang - 1 client · 128 docs per request: 1,304.1 docs/sPaddock - 1 client · 128 docs per request: 2,993.8 docs/s2.3×vLLM - 4 clients · 32 docs per request: 1,957.8 docs/sSGLang - 4 clients · 32 docs per request: 1,370.4 docs/sPaddock - 4 clients · 32 docs per request: 3,158.6 docs/s1.6×vLLM - 16 clients · 8 docs per request: 1,834.5 docs/sSGLang - 16 clients · 8 docs per request: 1,310.2 docs/sPaddock - 16 clients · 8 docs per request: 2,879.8 docs/s1.6×+1,690 docs/s

Qwen3-Reranker-4B

PaddockvLLMSGLang
docs/s02004006008001 client32 per request1 client128 per request4 clients32 per request16 clients8 per requestvLLM - 1 client · 32 docs per request: 320.9 docs/sSGLang - 1 client · 32 docs per request: 322.9 docs/sPaddock - 1 client · 32 docs per request: 583.7 docs/s1.8×vLLM - 1 client · 128 docs per request: 363.1 docs/sSGLang - 1 client · 128 docs per request: 427.1 docs/sPaddock - 1 client · 128 docs per request: 610.6 docs/s1.4×vLLM - 4 clients · 32 docs per request: 426.9 docs/sSGLang - 4 clients · 32 docs per request: 413.0 docs/sPaddock - 4 clients · 32 docs per request: 617.8 docs/s1.4×vLLM - 16 clients · 8 docs per request: 428.3 docs/sSGLang - 16 clients · 8 docs per request: 417.6 docs/sPaddock - 16 clients · 8 docs per request: 582.4 docs/s1.4×+261 docs/s

Qwen3-Reranker-8B

PaddockSGLang
docs/s01002003004001 client32 per request4 clients32 per request16 clients8 per requestSGLang - 1 client · 32 docs per request: 213.7 docs/sPaddock - 1 client · 32 docs per request: 326.2 docs/s1.5×SGLang - 4 clients · 32 docs per request: 244.0 docs/sPaddock - 4 clients · 32 docs per request: 340.5 docs/s1.4×SGLang - 16 clients · 8 docs per request: 240.1 docs/sPaddock - 16 clients · 8 docs per request: 329.1 docs/s1.4×+113 docs/s

Test Environment

Hardware

GPU
RTX 5090, 32 GB

Engine Versions

Paddock
pre-release build
vLLM
0.25.1. embed: --runner pooling. rerank: seq-classification via --hf-overrides {architectures:[Qwen3ForSequenceClassification]
SGLang
0.5.15.post1 (install --prerelease=allow)
llama.cpp
latest master @ 178a6c44 (~b10069)

Each engine ran the best setup it supports on this hardware, with everything on the GPU.

RTX PRO 6000 Blackwell · 2026-07-11

Up to 1.4x

vs vLLM

Faster in 12 of 15 tests

Up to 1.8x

vs vLLM (same-work)

Faster in 14 of 15 tests

Up to 1.5x

vs SGLang

Faster in 10 of 10 tests

Up to 11.4x

vs llama.cpp

Faster in 15 of 15 tests

We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.

Measured Throughput

Each engine ran the same tests, in docs/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was. vLLM appears twice here: its scoring API does less prompt work per document than Paddock by default, so the vLLM (same-work) rows also give it the identical full prompt. That is the like-for-like comparison.

PaddockvLLMvLLM (same-work)SGLangllama.cpp

Qwen3-Reranker-0.6B

vs vLLM: 0.8-1.4xvs vLLM (same-work): 1-1.8xvs SGLang: 1-1.5xvs llama.cpp: 7.2-11.4x
1 client · 32 docs per request
Paddock
907.5 docs/s
vLLM
642.31.41x
vLLM (same-work)
502.21.81x
SGLang
608.31.49x
llama.cpp
85.610.6x
1 client · 128 docs per request
Paddock
1,192.5 docs/s
vLLM
853.01.4x
vLLM (same-work)
749.51.59x
SGLang
957.11.25x
llama.cpp
104.711.39x
4 clients · 32 docs per request
Paddock
1,102.3 docs/s
vLLM
1,289.40.85x
vLLM (same-work)
1,082.21.02x
SGLang
1,030.41.07x
llama.cpp
148.97.4x
8 clients · 128 docs per request
Paddock
1,238.7 docs/s
vLLM
1,379.50.9x
vLLM (same-work)
1,233.01.0x
SGLang
1,189.01.04x
llama.cpp
148.28.36x
16 clients · 8 docs per request
Paddock
1,080.5 docs/s
vLLM
1,292.70.84x
vLLM (same-work)
1,110.20.97x
SGLang
1,016.51.06x
llama.cpp
149.67.22x

Qwen3-Reranker-4B

vs vLLM: 1.1-1.3xvs vLLM (same-work): 1.2-1.4xvs SGLang: 1.2-1.3xvs llama.cpp: 2.9-4.1x
1 client · 32 docs per request
Paddock
229.8 docs/s
vLLM
191.21.2x
vLLM (same-work)
163.21.41x
SGLang
178.51.29x
llama.cpp
59.93.84x
1 client · 128 docs per request
Paddock
264.2 docs/s
vLLM
211.21.25x
vLLM (same-work)
183.71.44x
SGLang
213.71.24x
llama.cpp
64.44.1x
4 clients · 32 docs per request
Paddock
260.2 docs/s
vLLM
223.61.16x
vLLM (same-work)
201.81.29x
SGLang
214.81.21x
llama.cpp
88.42.94x
8 clients · 128 docs per request
Paddock
262.4 docs/s
vLLM
229.71.14x
vLLM (same-work)
203.91.29x
SGLang
226.51.16x
llama.cpp
78.53.34x
16 clients · 8 docs per request
Paddock
252.7 docs/s
vLLM
224.91.12x
vLLM (same-work)
202.61.25x
SGLang
204.61.24x
llama.cpp
85.32.96x

Qwen3-Reranker-8B

vs vLLM: 1.1-1.2xvs vLLM (same-work): 1.3-1.4xvs llama.cpp: 2.2-2.9x
1 client · 32 docs per request
Paddock
130.7 docs/s
vLLM
109.01.2x
vLLM (same-work)
95.91.36x
llama.cpp
47.52.75x
1 client · 128 docs per request
Paddock
146.6 docs/s
vLLM
119.61.23x
vLLM (same-work)
105.71.39x
llama.cpp
50.02.93x
4 clients · 32 docs per request
Paddock
144.2 docs/s
vLLM
122.11.18x
vLLM (same-work)
109.41.32x
llama.cpp
64.12.25x
8 clients · 128 docs per request
Paddock
144.9 docs/s
vLLM
125.01.16x
vLLM (same-work)
110.91.31x
llama.cpp
62.62.31x
16 clients · 8 docs per request
Paddock
141.6 docs/s
vLLM
123.21.15x
vLLM (same-work)
109.61.29x
llama.cpp
62.82.25x

Throughput Under Load

The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.

Qwen3-Reranker-0.6B

PaddockvLLMvLLM (same-work)SGLangllama.cpp
docs/s03757501,1251,5001 client32 per request1 client128 per request4 clients32 per request8 clients128 per request16 clients8 per requestvLLM - 1 client · 32 docs per request: 642.3 docs/svLLM (same-work) - 1 client · 32 docs per request: 502.2 docs/sSGLang - 1 client · 32 docs per request: 608.3 docs/sllama.cpp - 1 client · 32 docs per request: 85.6 docs/sPaddock - 1 client · 32 docs per request: 907.5 docs/s1.4×vLLM - 1 client · 128 docs per request: 853.0 docs/svLLM (same-work) - 1 client · 128 docs per request: 749.5 docs/sSGLang - 1 client · 128 docs per request: 957.1 docs/sllama.cpp - 1 client · 128 docs per request: 104.7 docs/sPaddock - 1 client · 128 docs per request: 1,192.5 docs/s1.2×vLLM - 4 clients · 32 docs per request: 1,289.4 docs/svLLM (same-work) - 4 clients · 32 docs per request: 1,082.2 docs/sSGLang - 4 clients · 32 docs per request: 1,030.4 docs/sllama.cpp - 4 clients · 32 docs per request: 148.9 docs/sPaddock - 4 clients · 32 docs per request: 1,102.3 docs/s0.9×vLLM - 8 clients · 128 docs per request: 1,379.5 docs/svLLM (same-work) - 8 clients · 128 docs per request: 1,233.0 docs/sSGLang - 8 clients · 128 docs per request: 1,189.0 docs/sllama.cpp - 8 clients · 128 docs per request: 148.2 docs/sPaddock - 8 clients · 128 docs per request: 1,238.7 docs/s0.9×vLLM - 16 clients · 8 docs per request: 1,292.7 docs/svLLM (same-work) - 16 clients · 8 docs per request: 1,110.2 docs/sSGLang - 16 clients · 8 docs per request: 1,016.5 docs/sllama.cpp - 16 clients · 8 docs per request: 149.6 docs/sPaddock - 16 clients · 8 docs per request: 1,080.5 docs/s0.8×+265 docs/s

Qwen3-Reranker-4B

PaddockvLLMvLLM (same-work)SGLangllama.cpp
docs/s0751502253001 client32 per request1 client128 per request4 clients32 per request8 clients128 per request16 clients8 per requestvLLM - 1 client · 32 docs per request: 191.2 docs/svLLM (same-work) - 1 client · 32 docs per request: 163.2 docs/sSGLang - 1 client · 32 docs per request: 178.5 docs/sllama.cpp - 1 client · 32 docs per request: 59.9 docs/sPaddock - 1 client · 32 docs per request: 229.8 docs/s1.2×vLLM - 1 client · 128 docs per request: 211.2 docs/svLLM (same-work) - 1 client · 128 docs per request: 183.7 docs/sSGLang - 1 client · 128 docs per request: 213.7 docs/sllama.cpp - 1 client · 128 docs per request: 64.4 docs/sPaddock - 1 client · 128 docs per request: 264.2 docs/s1.2×vLLM - 4 clients · 32 docs per request: 223.6 docs/svLLM (same-work) - 4 clients · 32 docs per request: 201.8 docs/sSGLang - 4 clients · 32 docs per request: 214.8 docs/sllama.cpp - 4 clients · 32 docs per request: 88.4 docs/sPaddock - 4 clients · 32 docs per request: 260.2 docs/s1.2×vLLM - 8 clients · 128 docs per request: 229.7 docs/svLLM (same-work) - 8 clients · 128 docs per request: 203.9 docs/sSGLang - 8 clients · 128 docs per request: 226.5 docs/sllama.cpp - 8 clients · 128 docs per request: 78.5 docs/sPaddock - 8 clients · 128 docs per request: 262.4 docs/s1.1×vLLM - 16 clients · 8 docs per request: 224.9 docs/svLLM (same-work) - 16 clients · 8 docs per request: 202.6 docs/sSGLang - 16 clients · 8 docs per request: 204.6 docs/sllama.cpp - 16 clients · 8 docs per request: 85.3 docs/sPaddock - 16 clients · 8 docs per request: 252.7 docs/s1.1×+51 docs/s

Qwen3-Reranker-8B

PaddockvLLMvLLM (same-work)llama.cpp
docs/s038751131501 client32 per request1 client128 per request4 clients32 per request8 clients128 per request16 clients8 per requestvLLM - 1 client · 32 docs per request: 109.0 docs/svLLM (same-work) - 1 client · 32 docs per request: 95.9 docs/sllama.cpp - 1 client · 32 docs per request: 47.5 docs/sPaddock - 1 client · 32 docs per request: 130.7 docs/s1.2×vLLM - 1 client · 128 docs per request: 119.6 docs/svLLM (same-work) - 1 client · 128 docs per request: 105.7 docs/sllama.cpp - 1 client · 128 docs per request: 50.0 docs/sPaddock - 1 client · 128 docs per request: 146.6 docs/s1.2×vLLM - 4 clients · 32 docs per request: 122.1 docs/svLLM (same-work) - 4 clients · 32 docs per request: 109.4 docs/sllama.cpp - 4 clients · 32 docs per request: 64.1 docs/sPaddock - 4 clients · 32 docs per request: 144.2 docs/s1.2×vLLM - 8 clients · 128 docs per request: 125.0 docs/svLLM (same-work) - 8 clients · 128 docs per request: 110.9 docs/sllama.cpp - 8 clients · 128 docs per request: 62.6 docs/sPaddock - 8 clients · 128 docs per request: 144.9 docs/s1.2×vLLM - 16 clients · 8 docs per request: 123.2 docs/svLLM (same-work) - 16 clients · 8 docs per request: 109.6 docs/sllama.cpp - 16 clients · 8 docs per request: 62.8 docs/sPaddock - 16 clients · 8 docs per request: 141.6 docs/s1.1×+27 docs/s

Test Environment

Hardware

GPU
RTX PRO 6000 Blackwell, 96 GB
Driver
610.43.02
CUDA
13.0
OS
Linux 6.8.0-124

Engine Versions

Paddock
pre-release build
vLLM
0.24.0 (latest on PyPI at benchmark time)
SGLang
0.5.15 (latest on PyPI at benchmark time)
mistral.rs
v0.9.0 (latest release, 2026-07-07)
llama.cpp
b9967 (latest tag at benchmark time)

Each engine ran the best setup it supports on this hardware, with everything on the GPU.

Accuracy Checks

Paddock tunes how each model is served on this GPU. We only let it use a faster serving mode when the measured answer quality stays at or above its reference setup. For models where no faster mode passed that check, the benchmark ran on the reference setup. The per-model results are in the raw data.

How We Tested

  • Every engine ran on the same machine and GPU, never at the same time.
  • The same test program sent identical requests to each engine.
  • Every input was unique, so nothing could be served from a cache.
  • Each number is the best timed round, after a warmup.
  • We measure end to end: what an application connecting over HTTP actually gets.
  • Where the engines differ in ways that could affect the comparison, we note it alongside the results.

Check our numbers

Paddock ships a benchmark harness, so you can run the same comparison on your own hardware.

About PaddockGet Early Access