Truespar Paddock
Paddock is a low-latency, high-throughput serving engine for open-source language models on your own NVIDIA hardware. One executable with OpenAI and Anthropic compatible APIs, so most tools connect without any changes.
Why Paddock
Handles Real Traffic
Most local tools serve one request at a time. Paddock batches whatever arrives, so a coding agent hammering the server or ten people sharing one GPU both get full speed out of the card.
One Program
A single executable. There is no Python or PyTorch underneath, and no CUDA version to keep in sync. The GPU code ships inside the binary and adapts to your NVIDIA card when it starts. Windows and Linux.
Private by Default
Nothing is sent to anyone. Models, prompts and documents stay on your machine, which is the whole point for a lot of teams running AI in-house.
Measured Performance
Every number is end to end on the same machine, against the newest builds of llama.cpp, vLLM, SGLang and mistral.rs, with unique inputs so caching can't inflate anything. Each model has its own report with the raw data attached.
35 of 42
Tests faster than vLLM and SGLang
Gemma 4, Qwen3.5 and Qwen3.6, up to 4.0x
57 of 57
Tests faster than llama.cpp
Every published model, up to 15.9x
Up to 44x
Lower time to first token than llama.cpp
Gemma 4 31B at 8 concurrent clients
Built for Concurrent Traffic
Most of the load on an inference server today comes from software, not people. Agents carry long context, repeat the same prompts and fire many requests at once. Paddock is designed around that: continuous batching, chunked prefill, a paged KV cache with prefix caching, and speculative decoding.
Continuous Batching
Requests are batched as they arrive and long prompts are chunked, so prefill and decode share the same GPU ticks and the card never sits idle.
Prefix Caching
Repeated context is cached. When an agent sends the same system prompt or the same document for the hundredth time, Paddock skips the work it has already done.
Fair Scheduling
One caller can't starve the rest. Requests are scheduled fairly across users and agents, and speculative decoding with multi-token prediction speeds up generation on top.
OpenAI & Anthropic APIs
Chat Completions, the OpenAI Responses API and Anthropic Messages, with streaming, tool calling and structured outputs. Claude Code and most agent frameworks work as-is.
RAG in One Server
Embeddings, reranking and text generation run in the same server, so you can build search and Q&A over your own documents without any of it leaving the machine.
Text and Images
Vision models take images alongside text in the same request. And if something isn't supported, you get an error, not a silently wrong answer.
Supported Models
These are the models we support and test today. They load as plain GGUF files, the same format llama.cpp and Ollama use.
| Model | Type | Handles |
|---|---|---|
| gpt-oss-20b / 120b | Mixture-of-experts | Text |
| gemma-4-31B | Dense | Text + images |
| qwen3.5-9B | Dense hybrid | Text |
| qwen3.6-27B | Dense hybrid | Text + images |
| qwen3.6-35B-A3B | Mixture-of-experts | Text + images |
| Qwen3 Embedding (0.6B / 4B / 8B) | Text encoder | Embeddings |
| Qwen3 Reranker (0.6B / 4B / 8B) | Text encoder | Reranking |
Why It's Fast
The speed doesn't come from one trick. Paddock owns the whole path between a request and the silicon.
Custom CUDA Kernels, Tuned Per Chip
Every kernel is written in-house and tuned per NVIDIA generation, down to Blackwell's block-scaled FP4 tensor cores.
Continuous Batching
New requests join the running batch instead of waiting their turn, so the GPU stays busy and latency stays low even when many clients hit at once.
No Framework in the Way
Rust talking directly to CUDA. No Python, no PyTorch dispatch - everything between your request and the GPU earns its cost.
Paddock vs llama.cpp, Ollama, vLLM and SGLang
| Developer + production | Developer inference | Production inference | |||
|---|---|---|---|---|---|
| Feature | Paddock | llama.cpp | Ollama | vLLM | SGLang |
| Written in | Rust + CUDA | C/C++ + CUDA | Go, on llama.cpp's engine | Python + CUDA | Python + CUDA |
| Install | One executable | Executable + per-model flags | App installer | Python + containers | Python + containers |
| GPU support | NVIDIA only today | NVIDIA, AMD, Apple, CPU | NVIDIA, AMD, Apple | NVIDIA, AMD, Intel, more | NVIDIA, AMD, more |
| Windows and Linux | Both | Both | Both | Linux (WSL2) | Linux (WSL2) |
| Built for concurrent traffic | Continuous batching | Falls off under load in our tests | A few requests at once | Continuous batching | Continuous batching |
| Compatible APIs | OpenAI, Anthropic | OpenAI | Ollama, OpenAI | OpenAI | OpenAI |
| Accuracy-checked quantization | Calibrated per model | Your choice, unchecked | Your choice, unchecked | Your choice, unchecked | Your choice, unchecked |
| Web studio and benchmarking | Built in | Chat UI | Desktop app | None | None |
| Multi-node clusters | One box, by design | No | No | Yes | Yes |
Where these engines compete head to head, we measure it on the same GPU and publish the raw data: see the benchmarks.
How Paddock Is Built
Plain GGUF Files
Paddock loads the same GGUF files llama.cpp, Ollama and LM Studio use. No conversion step, no proprietary format. Your models stay yours.
Built-In Studio
A web UI for downloading and running models, and chatting with them with web search and MCP tools. Every run shows time to first token, tokens per second, and GPU load.
Free to Use
Free for individuals and companies, with nothing held back. The Enterprise plan adds SLA support and source code access, priced per GPU.
Frequently Asked Questions
Which GPUs does Paddock support?
NVIDIA GPUs from Ampere and newer - RTX 30-series, RTX A6000, Ada, and Blackwell - with the heaviest tuning on Blackwell. AMD and Apple silicon are not supported today.
Does Paddock run on Windows?
Yes. Paddock is one executable for Windows and Linux, with the GPU code inside the binary. There is no Python environment or container to set up.
Which models can Paddock serve?
gpt-oss 20b and 120b, Gemma 4 31B, Qwen 3.5 and 3.6, and the Qwen3 embedding and reranker families, loaded from plain GGUF files.
Which APIs does Paddock expose?
OpenAI Chat Completions and the Responses API, and Anthropic Messages, with streaming, tool calling and structured outputs. The built-in Studio adds MCP tools and web search in chat.
How fast is Paddock compared to vLLM, SGLang or llama.cpp?
On the newest models we published, Paddock was faster than vLLM and SGLang in 35 of 42 tests and faster than llama.cpp in 57 of 57. Every report and its raw data are on the benchmarks page.
Is Paddock free?
Yes, free for individuals and companies, with no usage limits. The Enterprise plan adds SLA support and source code access, priced per GPU per year.
Does anything leave my machine?
No. Models, prompts and documents stay on your hardware. Paddock sends nothing to anyone.
Try it on your hardware
Paddock is in research preview and built for NVIDIA GPUs; AMD and Apple silicon are not supported today. If you want early access, or want us to benchmark a GPU you care about, get in touch.