Getting Started

Paddock runs open-source models in production on your own NVIDIA hardware, with OpenAI-compatible and Anthropic-compatible APIs, for teams that need the inference to stay on their machines.

Install

Latest version: 0.1.3

# Download and extract
Invoke-WebRequest -Uri https://truespar.com/releases/paddock/latest/paddock-0.1.3-x86_64-windows.zip -OutFile paddock.zip
Expand-Archive paddock.zip -DestinationPath paddock

# Start the manager and the Studio
.\paddock\paddock.exe

# Download and extract
curl -LO https://truespar.com/releases/paddock/latest/paddock-0.1.3-x86_64-linux.tar.gz
tar xzf paddock-0.1.3-x86_64-linux.tar.gz
cd paddock

# Start the manager and the Studio
./paddock

Paddock is free of charge, with no usage limits, for personal and commercial use alike. The terms are the license, and the same text sits in LICENSE.txt inside the archive you just downloaded.

Paddock ships as two programs in one package: paddock, the manager and Studio, and paddock-runner, the model server it supervises.

Requirements

What Paddock Is

Paddock is a native Rust high-throughput inference engine and server. It has no dependencies except the NVIDIA driver. It runs on Windows and Linux, x64.

The server exposes both the OpenAI API (completions, and the Responses API) and the Anthropic Messages API, so existing SDKs and agent tools connect without changes. It also serves an embedded Studio web UI from the same binary, and OpenAI-style embedding and rerank endpoints.

Models are plain GGUF files, downloaded from our own object storage and checked against a built-in hash. For how fast it runs, see the benchmarks.

First Run

Paddock ships two programs. paddock is the manager: it runs the Studio, downloads models, supervises model servers, and owns GPU telemetry for the machine. paddock-runner is one model server. Start the manager:

paddock

That brings up the Studio on https://localhost:11500, where you can pick a model, watch it download and start it. Everything the Studio does is also a verb:

# download a catalog model and start serving it
paddock serve qwen3.6-27b

# what is running, with live health
paddock ps

serve takes a catalog id (for example qwen3.5-9b or qwen3.6-27b), an installed model name, or a path to a GGUF file. A catalog model that is not on disk yet is downloaded into the models directory and verified against a built-in SHA-256 hash, along with its vision tower or draft model when it has one.

Each model gets its own server on its own port, allocated upward from 11540, so several models can run side by side. paddock serve writes that server's configuration to ~/paddock/servers/<port>.toml; the file is yours to hand-edit, and paddock-runner --config will run it with no manager involved. Run paddock --help for all verbs, and see the CLI page for the full surface.

Send a Request

Any OpenAI or Anthropic client works by pointing its base URL at the model's port, not the manager's. With curl:

curl http://127.0.0.1:11540/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5-9b",
    "messages": [{"role": "user", "content": "Say hello."}]
  }'

The API surface each model server exposes:

A model server binds all interfaces by default and generates a Bearer key at startup, printed in the banner. Calls from the same machine never need it; calls over the network do. see Server Configuration.

Native FP8

Paddock also serves native FP8 on hardware that has it, reading the vendor's official FP8 or bf16 safetensors checkpoint directly instead of going through the Q8_0 middle step. Those checkpoints are not hosted with us, so fetch one yourself from the vendor and point the server at the directory:

paddock serve qwen3.6-27b --fp8-native ~/checkpoints/Qwen3.6-27B-FP8

The flag is --fp8-native <DIR> on a runner, or PADDOCK_FP8_NATIVE. It supplies the native weight planes over the GGUF you are already serving, so keep the model selection as it is and add the directory.

Next Steps

• Supported Models - the model families the engine serves today, including vision, speech and embedding models.
• CLI - the manager's verbs and the runner's flags.
• Server Configuration - flags, environment variables, the config file, and authentication.
• GPU Support - which GPUs Paddock serves, and which it refuses.
• Speech to Text - file and live transcription.
• Benchmarks - measured serving performance.
• License - what you may do with Paddock, and the full agreement.