Supported Models

Paddock implements each model family natively in the engine rather than shipping a generic loader over a huge model list. Every family below has its own hand-written path, CUDA on NVIDIA and Metal on Apple Silicon, and its own correctness gate.

How Support Works

Each release ships its model list compiled into the binary, so a given build only offers models it was built to load. There is no remote catalog to fetch and no version drift between the two: the download origin is a plain file host, and each file URL is stable once published. You can also point the server at any GGUF file on disk with --model.

The Weights column below is the default download. Most models offer a smaller alternative as well, and the Studio shows the choices with a VRAM estimate for each.

Those weights are the same on both platforms: the macOS build reads the same GGUF files on Metal that the NVIDIA build reads on CUDA. The MLX column is an extra option, not a requirement - where a model has an MLX package, macOS can load that instead. A dash there means GGUF only, on either platform.

Chat Models

Notes on individual families:

• GPT-OSS uses the Harmony output format; the server parses reasoning and tool calls natively, and reasoning effort (low, medium, high) is exposed through the API. The 120B is 117B parameters with 5.1B active per token, so it runs far faster than its size suggests, but the whole model still has to fit: native MXFP4 wants a large card, and a compact build exists for smaller ones.
• Gemma 4 26B A4B puts a shared dense feed-forward next to a 128-expert routed one on every layer, so only about 4B parameters are active per token. It reads images, and its answers are token-identical to the same weights on llama.cpp.
• Gemma 4 calls tools, on all three APIs. Its calls are read from Gemma's own call syntax rather than a generic one, stream as they are written, can be forced when a request requires a tool, and the turn carries on correctly after the results come back, pictures in them included. Until 0.1.18 those calls arrived as plain text, so an agent could not drive it. It also speculates on requests that declare tools, which they never used to, and it holds its embedding matrix once rather than twice - 0.78 GB less on the 26B-A4B and 1.5 GB on the 31B at Q8_0.
• Gemma 4 31B is the only family whose KV cache is 8-bit by default. Its speculative-decoding drafter is a separate 470M GGUF loaded with --mtp rather than a head built into the weights file. The engine now executes 4-bit k-quant Gemma 4 files such as Q4_K_M at 4 bits, so a compact GGUF you supply with --model runs as one; the download here is still the full-quality Q8_0.
• DiffusionGemma 26B A4B is the same 26B mixture of experts rewritten to write a block of text at once rather than one token after another, refining 256 words in parallel per step. Long answers arrive several times faster on the same card. It trails Gemma 4 26B on hard reasoning, and it is text only for now. It is also the model behind structured reads, which is what the parallel refinement is really for: every question in a set is answered in the same pass, with a probability attached.
• Bonsai 2 27B is Prism ML's reasoning model in ternary weights, on the same Qwen 3.8 hybrid backbone. The whole model is 5.9 GB, so there is very little to read per token and generation is quick for its size. Ternary weights are close to the original rather than identical, and there is no drafter, so no speculative decoding. Image input is a separate 0.9 GB vision download. On the DGX Spark its ternary weights run on the tensor cores at every batch size, with an 8-bit KV cache by default.
• MiniCPM5 2B is a 2.5B model for laptops and edge boxes: 2.7 GB at full quality, with a thinking mode you can switch off per request and tool calling with parallel calls. Text only.
• Qwen 3.5, 3.6 and 3.8 share one hybrid architecture in the engine: three Gated DeltaNet linear-attention layers for every full-attention layer. Dense and MoE variants load through the same code path, keyed by GGUF metadata, and the multi-token-prediction head is in the weights file, so speculative decoding needs no separate drafter. Qwen 3.8 27B is the same geometry as 3.6 27B, which is why it serves on the existing path rather than needing a bring-up of its own, and it will read a vendor NVFP4 or FP8 checkpoint you supply yourself alongside its Q8_0 download. Qwen 3.8 Flash Next is the one with an NVFP4 build of its own.
• Qwen 3.8 Flash Next attends sparsely past about 2,000 tokens, and Paddock serves it that way rather than falling back to full attention, which was not the model's own behaviour. The full 262K context loads, and a continued conversation resumes from its cache at any depth. Its cache is paged, so its memory is planned inside the endpoint's budget rather than taken in fixed amounts regardless of it, and a cached prefix is shared between turns instead of copied.
• Kolibri 1 is Aleph Alpha's German and English model for reasoning, retrieval and tool calling, with 3.46B of its 78B parameters active per token. It is a large download, 45 to 48 GB, and wants at least 64 GB of GPU or unified memory. On NVIDIA it serves a 4-bit GGUF on any generation, with the NVFP4 checkpoint offered on Blackwell; on Apple Silicon it serves a pinned mixed 4/8-bit MLX build, where the context defaults to 32K rather than the full 262K. Text only, and no speculative drafter. Its correctness is checked against Aleph Alpha's own serving code rather than llama.cpp, because no llama.cpp release reads these weights yet.
• Laguna is Poolside's coding and agent family. Attention repeats one full layer followed by three sliding-window layers, and every layer past the first is a mixture of experts with an always-on shared expert. Laguna S 2.1 carries a 1M context. Both take an optional DFlash draft model for extra speed.
• Granite is IBM's business-focused family: a dense stack tuned for tool calling and instruction following across twelve languages. Granite 4.1 Vision 4B is listed with the document readers rather than here, because that is what it is: it pulls the numbers out of a chart or a table in an image rather than chatting about photographs.
• Nemotron 3.5 Lightning is NVIDIA's hybrid: mostly Mamba layers with a few attention layers, so only 6 of its 52 layers page a KV cache. That is why it carries a 1M context without the memory cost one usually implies. It takes either of two optional draft models, DFlash or DSpark, and speculates on agent traffic as well as plain chat, including sampled and tool-calling requests at any concurrency. DSpark is NVIDIA's recommended drafter for the DGX Spark.
• Muse Glimmer 30B is Meta's Apache-2.0 reasoning model. It reasons before every answer, reads images, and ships its own DFlash draft model.

Document Readers

Specialists that read a scanned page, a photo of a document or a whole PDF rather than chatting about pictures. The first two take a reading mode on the request rather than a prompt describing one; see Document OCR.

• Unlimited-OCR reads whole PDFs, parses structure with the position of every block, and can point back at where on the page each piece of text came from. Two page-cutting strategies: a tiled close-up pass for fine print, and a faster whole-page pass for long documents.
• PaddleOCR-VL 1.6 is the compact one, with strong coverage of Nordic and other European languages, and dedicated modes for tables, formulas, charts and seals.

Speech to Text

Audio models are served through POST /v1/audio/transcriptions and the streaming WebSocket at GET /v1/realtime. See Speech to Text for the wire format.

• Whisper models are encoder-decoder and run on their own thread. All three are national-library fine-tunes, each substantially more accurate on its language than the original multilingual checkpoint.
• Qwen3-ASR and Granite Speech are generative: an audio tower feeds an ordinary language model. On Granite Speech the instruction picks the task, so the same checkpoint writes a punctuated transcript, translates, or biases recognition toward a keyword list, depending on what you put in prompt. The Plus variant adds speaker labels and word-level timings.

Speaker Diarization

Who spoke when, which is a different question from what was said. Served through POST /v1/audio/diarizations and a live WebSocket; see Speech to Text.

It finds up to eight speakers with 10 ms resolution and handles people talking over each other, which is where simpler approaches give up. It writes no words and names no people: speakers come back as numbers that mean something only within that one recording. Pair it with a transcription model for the words, or hand it a transcript and it will attribute each word to a speaker.

Embedding and Reranker Models

Encoder models are served through POST /v1/embeddings and POST /v1/rerank instead of the chat routes. The server detects an encoder architecture at load time and routes accordingly.

One of them is not text-only. EmbeddingGemma 2 puts text, pictures and audio into the same vector space, so a photo or a recording can be found with words, or a word with a photo. The text model is 0.3 GB; the picture tower (0.4 GB) is on by default and the audio tower (0.6 GB) is off, both switches on the start form, and the download follows what you pick. Vectors are 768 wide and can be truncated to 128 (Matryoshka). Video works too, on NVIDIA GPUs as well as Apple Silicon: each frame costs the 140 tokens the checkpoint is configured for, and a video that will not fit the 4,096-token context is refused with its token count rather than quietly cut short. Audio clips up to 30 seconds each.

At load time the server runs a short, deterministic quality calibration for encoder models to pick the fastest kernel profile that holds retrieval quality for that specific model. The result is cached on disk, so it costs a few seconds once per model and pack. It can be disabled with PADDOCK_BS_CALIB=off.

Image Input

Gemma 4 31B, Gemma 4 26B A4B, Qwen 3.5 9B, Qwen 3.6 27B, Qwen 3.6 35B A3B, Qwen 3.8 27B, Muse Glimmer 30B and Bonsai 2 27B accept image content parts in chat requests. The document readers take images too, for reading rather than for conversation. Image input needs the model's vision tower file alongside its weights, and the two download together for every family except Bonsai, whose tower is a separate 0.9 GB download.

Documents work on every model, not only the vision ones: a PDF becomes rendered pages for a model that can see and extracted text for one that cannot. See Documents and Images for how each file type is handled.

Table Models

A table model predicts a column from the rows around it. Nothing is trained: the labelled rows travel with every request. It is served through POST /v1/tabular/predictions; see Table Predictions.

Six checkpoints in all: three sizes, each built for either classification or regression. One endpoint serves one task. A classification answer is a class with probabilities, over up to ten classes in one pass and up to 256 through NVIDIA's error-correcting output codes; a regression answer carries 999 quantiles rather than a point estimate, so you get the spread and not just the middle.

Columns arrive as they are - numbers, categories, raw strings, missing cells - or already encoded if you would rather do your own feature engineering. Hundreds of rows take tens of milliseconds. It runs on NVIDIA GPUs and on Apple Silicon.

Decision Models

A decision model answers a fixed question about a text and does nothing else. It writes no words: every allowed answer is scored in one pass, and what comes back is a probability. It is served through POST /v1/systemone; see Structured Reads.

Clef and Clef Flash read a whole schema of questions at once and decide every field from one read of the state, so the answers are consistent with each other rather than arrived at separately. They take text, JSON or pictures, carry their own vision, and serve in 8-bit: Clef Flash wants about 16 GB of GPU memory and Clef about 37 GB with images. Both have an MLX package for Apple Silicon.

Clef is the larger of the two and ahead on intent and grounding work, but it is not ahead everywhere - Flash scores higher on MMLU and function calling - and it costs about four times as long a request. Start with Flash and move up only if the decision needs more knowledge than it has. Neither takes video, and neither supports the conditional questions the diffusion backend does.

Pictures are the one place to watch the clock: a 12 MP photo spends around 20 seconds in the vision tower, so send it smaller.

Laya is the fast end of the range: about 4 ms a question on a desktop GPU, because nothing is generated: one bidirectional pass scores every option at once. The 2.4 GB download holds three checkpoints - an English one, a multilingual one covering 100+ languages, and one for typed decisions - and Paddock picks between the first two from the text itself. A text longer than one window is read in overlapping windows rather than quietly cut.

It is worth knowing what it is not. Laya is weak on a task it has seen no examples of: on the vendor's own typed-decisions benchmark the base checkpoints trail the majority-class baseline. Accuracy falls off past about 20 options. It reads text only, runs on NVIDIA GPUs only, and its calibration on a choice with more than ten options is off, because the English checkpoint ships an out-of-range temperature there, which Paddock clamps as the reference does.

A runner serving Laya serves nothing else: no chat, no embeddings, just the one endpoint. DiffusionGemma answers the same endpoint with more it can do, and the two are compared on the Structured Reads page.

Segmentation

Name a thing and get back every one of them in the picture, each with a mask. Served through POST /v1/masks; see Segmentation.

The engine reads a SAM 3.1 checkpoint as well, telling the two apart from the weights, though video tracking needs SAM 3. One request finds every instance of a concept rather than the one object you clicked, and asking a second question about the same picture skips the encoding, so it comes back in milliseconds. It takes a short noun phrase, example boxes, or both, with negative boxes to say what to leave out. It finds things; it does not reason about them.

Aerial Segmentation

Dense prediction over an area rather than a question about one picture: every square metre gets a class and a height. Served through POST /v1/segmentations; see Aerial Segmentation.

Land cover and canopy height from one pass, both at 1 m, from Swedish aerial photography. It reads the near-infrared band rather than RGB alone, which is what separates vegetation from things that merely look green. Trained on 2,743 ten-kilometre cells across 17 counties and scored on four it never saw.

It is built for Swedish imagery at 0.5 m in SWEREF 99 TM and is not a general-purpose aerial model. Deciduous forest and open land are its weak classes, at 0.55 and 0.58 IoU against 0.81 to 0.93 elsewhere. Chips go through sixteen at a time, so it is built for sweeping an area rather than reading one image.

Image Generation

One model makes pictures rather than reading them. It is served through its own endpoints, not the chat routes; see Image Generation.

The prompt is read by a full 8B language model rather than a short text encoder, so long descriptions hold together, and text inside the picture (signs, labels, posters) comes out legible. Any size on a 32-pixel grid is allowed, transparent backgrounds work, and every render is seeded and reproducible.

It is three downloads: the diffusion weights, the text encoder and the VAE. Full quality wants about 17 GB of VRAM and the compact build about 10 GB, plus working room that grows with the picture - roughly 6 GB more at 1024 x 1024. A 1024 x 1024 image at the model's 40 steps takes about 40 seconds on an A6000. Editing a reference image needs the optional vision tower alongside.

GGUF Loading and Quantization

Paddock loads standard GGUF files, memory-mapped from disk, including multi-part files. The formats the engine executes today:

The k-quant formats got faster in 0.1.18: the kernel that reads prompts by 10 to 17%, and the small batches that check speculative drafts through a new tensor-core path. Under a twelve-session Claude Code load on Qwen 3.8 27B at 4 bits, decode went from 31.3 to 37.5 tokens a second per session on an RTX 5090-class card, with the same results to the bit.

Q8_0, the Q4 variants and NVFP4 are published by us and arrive with the model, hash-checked, in one click. Which compact option a model offers depends on the model: NVFP4 where one exists, and a Q4 k-quant where it does not.

Apple Silicon

The macOS build runs on Metal rather than CUDA, and it serves the same catalog from the same GGUF files. A model does not need an MLX package to run on a Mac; the engine's Metal graphs read GGUF directly, k-quants included.

Where an MLX package does exist - Gemma 4 31B, Qwen 3.8 27B, Qwen 3.8 Flash Next, DiffusionGemma 26B A4B, Muse Glimmer 30B, MiniCPM5 2B, Bonsai 2 27B and Qwen-Image 2.1 - it is an alternative download for Apple Silicon, in the MLX column above. Bonsai's MLX package bundles its vision tower, where the GGUF build keeps it as a separate file.

macOS builds are published on GitHub releases rather than through the updater here, so the macOS version can differ from the Windows and Linux one.

FP8 is the one exception. It comes from the vendor's own safetensors checkpoint rather than a GGUF conversion, and we do not host those, so you fetch one yourself and point the server at the directory with --fp8-native. See Getting Started.

For measured serving throughput per model, see the benchmarks.