Supported Models
Paddock implements most common and newer model families natively rather than a huge library of models.
How Support Works
Every model family in Paddock has its own native tuning in the engine, running CUDA kernels. Each release contains a built-in catalog of models, downloaded from our fast object storage.
Generative Models
| Model | Architecture | Modalities | Weights |
|---|---|---|---|
| GPT-OSS 20B | Mixture of Experts (32 experts) | Text | MXFP4 |
| GPT-OSS 120B | Mixture of Experts (128 experts) | Text | MXFP4 |
| Qwen3.5 9B | Dense hybrid (Gated DeltaNet + gated attention) | Text | Q8_0 |
| Qwen3.6 27B | Dense hybrid | Text + images | Q8_0 |
| Qwen3.6 35B A3B | Hybrid Mixture of Experts (256 experts) | Text + images | Q8_0 |
| Gemma 4 31B | Dense (interleaved sliding-window and global attention) | Text + images | Q8_0 |
Notes on individual families:
- GPT-OSS models use the Harmony output format; the server parses reasoning and tool calls natively, and reasoning effort (low, medium, high) is exposed through the API.
- Qwen3.5 and Qwen3.6 share one hybrid architecture in the engine: three Gated DeltaNet linear-attention layers for every full-attention layer. The dense and MoE variants load through the same code path, keyed by GGUF metadata.
- Qwen3.6 35B A3B ships a built-in multi-token-prediction (MTP) head in its GGUF, which the engine uses for speculative decoding.
- Gemma 4 31B is the newest family in the engine. Its speculative-decoding drafter is a separate small GGUF loaded with the
--mtpflag rather than a built-in head.
Embedding and Reranker Models
Encoder models are served through POST /v1/embeddings and POST /v1/rerank instead of the chat routes. The server detects an encoder architecture at load time and routes accordingly.
| Model | Task | Weights |
|---|---|---|
| Qwen3 Embedding 0.6B / 4B / 8B | Text embeddings (Matryoshka dimensions) | Q8_0 |
| Qwen3 Reranker 0.6B / 4B / 8B | Query-document relevance scoring | Q8_0 |
At load time the server runs a short, deterministic quality calibration for encoder models to pick the fastest kernel profile that holds retrieval quality for that specific model. The verdict is cached on disk, so it costs a few seconds once per model and pack. It can be disabled with PADDOCK_BS_CALIB=off.
Vision Input
Qwen3.6 27B, Qwen3.6 35B A3B, and Gemma 4 accept image content parts in chat requests.
When a pdfium library is configured, the server also accepts PDF attachments on the vision path: pages are rasterized server-side into images, with a page cap. See Server Configuration for the PDF settings.
GGUF Loading and Quantization
Paddock loads standard GGUF files, memory-mapped from disk. The formats the engine executes today:
| Format | Status | Used by |
|---|---|---|
| Q8_0 | Executed on GPU | Qwen3.5 / Qwen3.6 / Gemma 4 / Qwen3 encoders |
| MXFP4 | Executed on GPU | GPT-OSS expert weights |
| F32 / F16 / BF16 | Executed on GPU | Norms, embeddings, vision towers |
For measured serving throughput per model, see the benchmarks.