Getting Started

Paddock is a high-throughput LLM inference server for NVIDIA GPUs, with OpenAI-compatible and Anthropic-compatible APIs.

What Paddock Is

Paddock is a native Rust high-throughput inference engine and server that ships as one binary. It has no dependencies except the NVIDIA driver. It runs on Windows and Linux.

The server exposes both the OpenAI API (completions, and the Responses API) and the Anthropic Messages API, so existing SDKs and agent tools connect without changes. It also serves an embedded Studio web UI from the same binary, and OpenAI-style embedding and rerank endpoints.

Models are plain GGUF files that are downloaded from our fast object storage. For how fast it runs, see the benchmarks.

Research preview. Paddock is not yet publicly downloadable. If you want early access, contact us and tell us about your hardware and workload.

What You Need

RequirementDetail
GPUAn NVIDIA GPU of the Ampere generation or newer. See GPU Support for the exact architecture list.
DriverAn NVIDIA driver that supports CUDA 13. The engine is built against the CUDA 13 API; you do not need to install the CUDA toolkit.
Operating systemWindows or Linux. AMD GPUs and Apple silicon are not supported today.
DiskSpace for the GGUF model files you want to serve. The current catalog ranges from under 1 GB for the small encoder models to about 38 GB for the largest generative model.

First Run

The server binary is called paddock. Point it at a model and the CUDA kernel pack that ships with the release:

paddock --model qwen3.5-9b --device cuda --kernel-pack <path-to-kernel-pack>

The --model value can be a file path to a GGUF, or a bare model id from the release's built-in catalog (for example qwen3.5-9b or qwen3.6-27b). A catalog id that is not on disk yet is downloaded automatically into the models directory and verified against a built-in SHA-256 hash. For vision models the catalog also fetches the matching vision tower (mmproj) file and loads it.

By default the server binds to 127.0.0.1:11540 and scans ~/paddock/models for GGUF files. On startup it prints a banner with the API address, the loaded model, the models directory, and the effective configuration sources. Run paddock --help for all options, and see Server Configuration for the full configuration surface.

Starting without --model is valid: the server comes up with no model loaded and serves GET /v1/models so you can browse what is installed.

Send a Request

Any OpenAI or Anthropic client works by pointing its base URL at the server. With curl:

curl http://127.0.0.1:11540/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5-9b",
    "messages": [{"role": "user", "content": "Say hello."}]
  }'

The API surface on port 11540:

EndpointDescription
GET /healthzLiveness check.
GET /v1/modelsInstalled models, OpenAI list shape.
POST /v1/chat/completionsOpenAI chat completions.
POST /v1/completionsOpenAI text completions.
POST /v1/responsesOpenAI Responses API.
POST /v1/messagesAnthropic Messages API.
POST /v1/messages/count_tokensAnthropic token counting.
POST /v1/embeddingsEmbeddings (encoder models).
POST /v1/rerankQuery-document reranking (encoder models).

On the default loopback bind no API key is required. If you bind to a non-loopback address the server generates and requires a Bearer key automatically; see Server Configuration.

Next Steps

  • Supported Models - the model families the engine serves today, including vision and embedding models.
  • Server Configuration - flags, environment variables, the paddock.toml config file, and authentication.
  • GPU Support - which GPU architectures are supported and which are performance-tuned.
  • Benchmarks - measured serving performance.