Supported Models

Paddock implements each model family natively in the engine rather than shipping a generic loader over a huge model list. Every family below has its own CUDA path and its own correctness gate.

How Support Works

Each release ships its model list compiled into the binary, so a given build only offers models it was built to load. There is no remote catalog to fetch and no version drift between the two: the download origin is a plain file host, and each file URL is stable once published. You can also point the server at any GGUF file on disk with --model.

The Weights column below is the default download. Most models offer a smaller alternative as well, and the Studio shows the choices with a VRAM estimate for each.

Chat Models

Notes on individual families:

• GPT-OSS uses the Harmony output format; the server parses reasoning and tool calls natively, and reasoning effort (low, medium, high) is exposed through the API. The 120B is 117B parameters with 5.1B active per token, so it runs far faster than its size suggests, but the whole model still has to fit: native MXFP4 wants a large card, and a compact build exists for smaller ones.
• Gemma 4 26B A4B puts a shared dense feed-forward next to a 128-expert routed one on every layer, so only about 4B parameters are active per token. It reads images, and its answers are token-identical to the same weights on llama.cpp.
• Gemma 4 31B is the only family whose KV cache is 8-bit by default. Its speculative-decoding drafter is a separate 470M GGUF loaded with --mtp rather than a head built into the weights file.
• Qwen 3.5, 3.6 and 3.8 share one hybrid architecture in the engine: three Gated DeltaNet linear-attention layers for every full-attention layer. Dense and MoE variants load through the same code path, keyed by GGUF metadata, and the multi-token-prediction head is in the weights file, so speculative decoding needs no separate drafter. Qwen 3.8 27B is the same geometry as 3.6 27B, which is why it serves on the existing path rather than needing a bring-up of its own. It is also the first model with an NVFP4 build alongside the default Q8_0, and it will read a vendor FP8 checkpoint you supply yourself.
• Laguna is Poolside's coding and agent family. Attention repeats one full layer followed by three sliding-window layers, and every layer past the first is a mixture of experts with an always-on shared expert. Laguna S 2.1 carries a 1M context. Both take an optional DFlash draft model for extra speed.
• Granite is IBM's business-focused family: a dense stack tuned for tool calling and instruction following across twelve languages. Granite 4.1 Vision 4B is listed with the document readers rather than here, because that is what it is: it pulls the numbers out of a chart or a table in an image rather than chatting about photographs.
• Nemotron 3.5 Lightning is NVIDIA's hybrid: mostly Mamba layers with a few attention layers, so only 6 of its 52 layers page a KV cache. That is why it carries a 1M context without the memory cost one usually implies, and why a long conversation does not get slower the further it goes.
• Muse Glimmer 30B is Meta's Apache-2.0 reasoning model. It reasons before every answer, reads images, and ships its own DFlash draft model.

Document Readers

Specialists that read a scanned page, a photo of a document or a whole PDF rather than chatting about pictures. The first two take a reading mode on the request rather than a prompt describing one; see Document OCR.

• Unlimited-OCR reads whole PDFs, parses structure with the position of every block, and can point back at where on the page each piece of text came from. Two page-cutting strategies: a tiled close-up pass for fine print, and a faster whole-page pass for long documents.
• PaddleOCR-VL 1.6 is the compact one, with strong coverage of Nordic and other European languages, and dedicated modes for tables, formulas, charts and seals.

Speech to Text

Audio models are served through POST /v1/audio/transcriptions and the streaming WebSocket at GET /v1/realtime. See Speech to Text for the wire format.

• Whisper models are encoder-decoder and run on their own thread. All three are national-library fine-tunes, each substantially more accurate on its language than the original multilingual checkpoint.
• Qwen3-ASR and Granite Speech are generative: an audio tower feeds an ordinary language model. On Granite Speech the instruction picks the task, so the same checkpoint writes a punctuated transcript, translates, or biases recognition toward a keyword list, depending on what you put in prompt. The Plus variant adds speaker labels and word-level timings.

Embedding and Reranker Models

Encoder models are served through POST /v1/embeddings and POST /v1/rerank instead of the chat routes. The server detects an encoder architecture at load time and routes accordingly.

At load time the server runs a short, deterministic quality calibration for encoder models to pick the fastest kernel profile that holds retrieval quality for that specific model. The result is cached on disk, so it costs a few seconds once per model and pack. It can be disabled with PADDOCK_BS_CALIB=off.

Image Input

Gemma 4 31B, Gemma 4 26B A4B, Qwen 3.5 9B, Qwen 3.6 27B, Qwen 3.6 35B A3B, Qwen 3.8 27B and Muse Glimmer 30B accept image content parts in chat requests. The document readers take images too, for reading rather than for conversation. Image input needs the model's vision tower file alongside its weights, and the two download together.

Documents work on every model, not only the vision ones: a PDF becomes rendered pages for a model that can see and extracted text for one that cannot. See Documents and Images for how each file type is handled.

GGUF Loading and Quantization

Paddock loads standard GGUF files, memory-mapped from disk, including multi-part files. The formats the engine executes today:

Q8_0, the Q4 variants and NVFP4 are published by us and arrive with the model, hash-checked, in one click. Which compact option a model offers depends on the model: NVFP4 where one exists, and a Q4 k-quant where it does not.

FP8 is the one exception. It comes from the vendor's own safetensors checkpoint rather than a GGUF conversion, and we do not host those, so you fetch one yourself and point the server at the directory with --fp8-native. See Getting Started.

For measured serving throughput per model, see the benchmarks.