Truespar Paddock
Production inference on your own NVIDIA hardware, with OpenAI- and Anthropic-compatible APIs, so most tools connect without any changes.
Who Paddock is for
Companies and public sector bodies that run inference on their own hardware, whether for privacy, sovereignty or regulation, and want datacenter-grade serving.
That means continuous batching, prefix caching and speculative decoding, on the APIs your existing tools already use.
Context is enriched with what the raw text leaves out, and kept small: tool definitions load on demand, repeated context is cached, and a conversation that outgrows the window is trimmed or compacted automatically.
Why Paddock
It's Fast
Most local tools serve one request at a time. Paddock batches whatever arrives, so an agent hammering the server or ten people sharing one GPU both get full speed out of the card.
No Dependencies
Unzip a folder and run it. There is no Python or PyTorch underneath, and no CUDA version to keep in sync. Tuned CUDA kernels for your NVIDIA card when it starts. Windows and Linux.
Private by Default
Nothing is sent to anyone. Models, prompts and documents stay on your machine.
Measured Performance
Every engine runs on the same GPU, one at a time, driven by the same client, on the prompt shapes the industry publishes. Each one is measured against the Paddock configuration on the same weights it runs. Numbers below are Qwen3.8 27B on RTX PRO 6000, 2026-08-22/23.
13 of 13
Scenarios faster than vLLM
Identical FP8 checkpoint, 1.02x to 1.19x
10 of 13
Scenarios faster than SGLang
Identical FP8 checkpoint, up to +9.4%, behind on 2
13 of 13
Scenarios faster than llama.cpp
Identical GGUF file, 1.5x to 37.5x
Built for Concurrent Traffic
Most of the load on an inference server today comes from software, not people. Agents carry long context, repeat the same prompts and fire many requests at once. Paddock is designed around that: continuous batching, chunked prefill, a paged KV cache with prefix caching, and speculative decoding.
Continuous Batching
Requests are batched as they arrive and long prompts are chunked, so prefill and decode share the same GPU ticks and the card never sits idle.
Prefix Caching
Repeated context is cached. When an agent sends the same system prompt or the same document for the hundredth time, Paddock skips the work it has already done.
Fair Scheduling
One caller can't starve the rest. Requests are scheduled fairly across users and agents, and speculative decoding with multi-token prediction speeds up generation on top.
OpenAI & Anthropic APIs
Chat Completions, the OpenAI Responses API and Anthropic Messages, with streaming, tool calling and structured outputs. Claude Code and most agent frameworks work as is.
Documents That Just Work
Attach a PDF, a spreadsheet or a photo and it becomes something the model can read: rendered pages for a model that sees, extracted text and metadata for one that doesn't. Embeddings and reranking run in the same server, so search over your own documents never leaves the machine.
Text, Images and Speech
Vision models take images alongside text in the same request, and speech models turn a recording or a live microphone into text on the same server. If something isn't supported, you get an error, not a silently wrong answer.
Supported Models
These are the models we implement natively and test today. They load as plain GGUF files, the same format llama.cpp and Ollama use.
| Provider | Model | Type | Handles |
|---|---|---|---|
| Alibaba | Qwen 3.5 9B | Dense hybrid | Text + images |
| Alibaba | Qwen 3.6 27B | Dense hybrid | Text + images |
| Alibaba | Qwen 3.6 35B A3B | Mixture-of-experts | Text + images |
| Alibaba | Qwen 3.8 27B | Dense hybrid | Text + images |
| Alibaba | Qwen3-ASR 1.7B | Audio tower + LLM | Speech to text |
| Alibaba | Qwen 3 Embedding 0.6B / 4B / 8B | Text encoder | Embeddings |
| Alibaba | Qwen 3 Reranker 0.6B / 4B / 8B | Text encoder | Reranking |
| Alibaba | Qwen3 Forced Aligner 0.6B | Audio tower + LLM | Word timings for a transcript |
| Baidu | Unlimited-OCR | Document reader | Pages and PDFs to text |
| CoRal Project | Røst v3 Whisper 1.5B | Encoder-decoder | Speech to text, Danish |
| Gemma 4 31B | Dense | Text + images | |
| Gemma 4 26B A4B | Hybrid dense and MoE | Text + images | |
| IBM | Granite 4.1 8B | Dense | Text |
| IBM | Granite 4.1 30B | Dense | Text |
| IBM | Granite 4.1 Vision 4B | Dense + image tower | Text + documents |
| IBM | Granite Speech 4.1 2B | Audio tower + LLM | Speech to text |
| IBM | Granite Speech 4.1 2B Plus | Audio tower + LLM | Speech to text, speaker labels and word timings |
| KBLab | KB-Whisper Large | Encoder-decoder | Speech to text, Swedish |
| Meta | Muse Glimmer 30B | Dense | Text + images |
| NB AI-Lab | NB-Whisper Large | Encoder-decoder | Speech to text, Norwegian |
| NVIDIA | Nemotron 3.5 Lightning 30B-A3B | Hybrid Mamba and attention | Text, 1M context |
| OpenAI | GPT-OSS 20B | Mixture-of-experts | Text |
| OpenAI | GPT-OSS 120B | Mixture-of-experts | Text |
| PaddlePaddle | PaddleOCR-VL 1.6 | Document reader | Pages to text, tables, formulas |
| Poolside | Laguna XS 2.1 | Mixture-of-experts | Text |
| Poolside | Laguna S 2.1 | Mixture-of-experts | Text, 1M context |
Architecture, context length and weights for each one are on the Supported Models page.
Why It's Fast
The speed doesn't come from one trick. Paddock owns the whole path between a request and the GPU.
Custom CUDA Kernels, Tuned Per GPU
Every kernel is written in-house and tuned per NVIDIA generation, down to Blackwell's block-scaled FP4 tensor cores.
Continuous Batching
New requests join the running batch instead of waiting their turn, so the GPU stays busy and latency stays low even when many clients hit at once.
No Framework in the Way
Rust talking directly to CUDA. No Python, no PyTorch, nothing in between.
Paddock vs llama.cpp, Ollama, vLLM and SGLang
| Feature | Paddock | llama.cpp | Ollama | vLLM | SGLang |
|---|---|---|---|---|---|
| Written in | Rust | C++ | Go | Python | Python |
| Dependencies | None | None | None | Python or a container | Python or a container |
| Runs on Windows | Yes | Yes | Yes | Through WSL2 | Through WSL2 |
| Hardware | NVIDIA only | NVIDIA, AMD, Apple, CPU | NVIDIA, AMD, Apple, CPU | NVIDIA, AMD, Intel, TPU and more | NVIDIA, AMD, Intel, TPU and more |
| APIs | OpenAI and Anthropic | OpenAI | Ollama and OpenAI | OpenAI and Anthropic | OpenAI |
| Web / Studio | Yes | Yes | Yes | No | No |
| Compare two models side by side | Yes | No | No | No | No |
| Cloud models beside local ones | OpenAI, Anthropic, OpenRouter, any endpoint | No | No | No | No |
| Reads PDF, Word and Excel | Yes | No | No | No | No |
| Context enrichment | Metadata, photo location, forensics | No | No | No | No |
Where these engines compete, we measure it on the same GPU, on identical weights: see the benchmarks.
How Paddock Is Built
Plain GGUF Files
Paddock loads the same GGUF files llama.cpp, Ollama and LM Studio use. No conversion step, no proprietary format. Your models stay yours.
Compare Models Side by Side
Ask one question and send it to several models at once, then read the answers next to each other. A model on your own GPU against a hosted one on your own key, or two local models, or the same model at two settings. Each answer carries its own time to first token, tokens per second and GPU load, so you decide on numbers rather than impressions. It is all in the built-in Studio, which also downloads and runs models and gives them web search and MCP tools.
Free, and Opening
Free for individuals and companies, with no usage limits. Paddock goes open source under MIT or Apache-2.0 during 2026, once it has run in extensive production testing. How that works.
Frequently Asked Questions
Which GPUs does Paddock support?
Blackwell cards (RTX PRO, GeForce RTX 50, B200) and Ampere workstation and consumer cards (RTX A6000, GeForce RTX 30, A40, A10). Paddock serves a GPU generation only once it has been tuned and benchmarked on that hardware, so other NVIDIA generations are refused at startup rather than served at unknown speed. AMD and Apple hardware are not supported.
Does Paddock run on Windows?
Yes, and on Linux. Both are x64, with the GPU code inside the binary and no Python environment or container to set up.
Which models can Paddock serve?
Chat and vision models from OpenAI, Google, Alibaba, Poolside, IBM and Meta, document readers from Baidu and PaddlePaddle, six speech-to-text models plus a forced aligner, and the Qwen3 embedding and reranker families. Everything loads from plain GGUF files, including your own.
Which APIs does Paddock expose?
OpenAI Chat Completions, the Responses API, embeddings, reranking and audio transcription, plus Anthropic Messages, with streaming, tool calling and structured outputs. Live microphone transcription runs over the Realtime WebSocket. The built-in Studio adds MCP tools and web search in chat.
How fast is Paddock compared to vLLM, SGLang or llama.cpp?
On our newest published board (Qwen3.8 27B, RTX PRO 6000), Paddock was faster on 13 of 13 test scenarios than vLLM, 10 of 13 than SGLang and 13 of 13 than llama.cpp. The full board is on the benchmarks page.
Can I compare a local model against a hosted one?
Yes. Add an OpenAI, Anthropic or OpenRouter account with your own key, or any OpenAI-compatible endpoint, and those models join the same picker as your local ones. A single turn can ask both and show the answers side by side. Keys stay on the server and are never sent to the browser.
Is Paddock free?
Yes. Free to run in development and production, for individuals and companies, with no usage limits and nothing held back for a paid tier. Paddock goes open source under MIT or Apache-2.0 during 2026, once it has run in extensive production testing. Support agreements and engineering time are how we fund the work.
Does anything leave my machine?
Nothing, unless you switch it on. Models, prompts and documents stay on your hardware, and Paddock contacts no service of ours to run. Three features reach outside because they cannot work otherwise, each off until you enable it with your own account: web search, cloud models, and the optional map view for a geotagged photo, which names the host before it calls it. Everything else, including reading where a photo was taken, happens on your own machine.
Try it on your hardware
Paddock is free to download and built for NVIDIA GPUs; AMD and Apple hardware are not supported. Windows and Linux, x64. If you want us to benchmark a GPU you care about, get in touch.