Qwen3 Embedding
Paddock vs vLLM vs SGLang vs llama.cpp · Embeddings · RTX 5090 (32 GB) · 2026-07-20
Qwen3 text-embedding model for retrieval, classification and ranking across 100+ languages.
RTX 5090 · 2026-07-20
Up to 3.5x
vs vLLM
Faster in 13 of 13 tests
Up to 6.3x
vs SGLang
Faster in 13 of 13 tests
Up to 18.7x
vs llama.cpp
Faster in 13 of 13 tests
We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.
Measured Throughput
Each engine ran the same tests, in texts/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was.
Qwen3-Embedding-0.6B
Qwen3-Embedding-4B
Qwen3-Embedding-8B
Throughput Under Load
The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.
Qwen3-Embedding-0.6B
Qwen3-Embedding-4B
Qwen3-Embedding-8B
Test Environment
Hardware
- GPU
- RTX 5090, 32 GB
Engine Versions
- Paddock
- pre-release build
- vLLM
- 0.25.1. embed: --runner pooling. rerank: seq-classification via --hf-overrides {architectures:[Qwen3ForSequenceClassification]
- SGLang
- 0.5.15.post1 (install --prerelease=allow)
- llama.cpp
- latest master @ 178a6c44 (~b10069)
Each engine ran the best setup it supports on this hardware, with everything on the GPU.
RTX PRO 6000 Blackwell · 2026-07-11
Up to 2.1x
vs vLLM
Faster in 15 of 15 tests
Up to 3x
vs SGLang
Faster in 15 of 15 tests
Up to 6.3x
vs mistral.rs
Faster in 15 of 15 tests
Up to 15.9x
vs llama.cpp
Faster in 15 of 15 tests
We measure throughput end to end and take the best timed round after a warmup. Each model is tested several ways, from one client sending large requests to many clients hitting the server at once. Every input is unique, so caching can't inflate anyone's numbers.
Measured Throughput
Each engine ran the same tests, in texts/s. Longer bars are better; the number next to a competitor is how many times faster Paddock was.
Qwen3-Embedding-0.6B
Qwen3-Embedding-4B
Qwen3-Embedding-8B
Throughput Under Load
The same results drawn as lines, from a single client on the left to the most concurrent test on the right. The shaded area is the gap between Paddock and the strongest competitor at each point, and the small numbers are Paddock's speedup there.
Qwen3-Embedding-0.6B
Qwen3-Embedding-4B
Qwen3-Embedding-8B
Test Environment
Hardware
- GPU
- RTX PRO 6000 Blackwell, 96 GB
- Driver
- 610.43.02
- CUDA
- 13.0
- OS
- Linux 6.8.0-124
Engine Versions
- Paddock
- pre-release build
- vLLM
- 0.24.0 (latest on PyPI at benchmark time)
- SGLang
- 0.5.15 (latest on PyPI at benchmark time)
- mistral.rs
- v0.9.0 (latest release, 2026-07-07)
- llama.cpp
- b9967 (latest tag at benchmark time)
Each engine ran the best setup it supports on this hardware, with everything on the GPU.
Accuracy Checks
Paddock tunes how each model is served on this GPU. We only let it use a faster serving mode when the measured answer quality stays at or above its reference setup. For models where no faster mode passed that check, the benchmark ran on the reference setup. The per-model results are in the raw data.
How We Tested
- Every engine ran on the same machine and GPU, never at the same time.
- The same test program sent identical requests to each engine.
- Every input was unique, so nothing could be served from a cache.
- Each number is the best timed round, after a warmup.
- We measure end to end: what an application connecting over HTTP actually gets.
- Where the engines differ in ways that could affect the comparison, we note it alongside the results.
Check our numbers
Paddock ships a benchmark harness, so you can run the same comparison on your own hardware.