Metrics
Each model server exposes GET /metrics in Prometheus format. Point a scraper at a runner's port and you get request latency, token throughput, KV cache occupancy and prefix-cache effectiveness for that model.
Naming
Where OpenTelemetry's GenAI semantic conventions define a metric, Paddock uses their name under the standard mapping, on the specification's own bucket boundaries. Where the concept is an engine internal with no convention to follow, the name is paddock_-prefixed, the way vLLM prefixes its own.
The GenAI server metrics are still marked Development stability upstream. That is accepted deliberately: if they change, what follows is a rename rather than a redesign, and a shared vocabulary now is worth more than a private one that never moves.
Semantic-convention histograms
| Metric | What it measures |
|---|---|
gen_ai_server_request_duration_seconds | End-to-end request time |
gen_ai_server_time_to_first_token_seconds | Time to first token |
gen_ai_server_time_per_output_token_seconds | Inter-token time during generation |
All three share a 14-boundary ladder that doubles from 10 ms. The Studio's charts and the server's own rollups use the same boundaries, so a percentile read from a dashboard and one read from a scrape cannot disagree about shape.
Engine counters and gauges
| Metric | What it tells you |
|---|---|
paddock_prompt_tokens, paddock_generation_tokens | Token throughput in and out |
paddock_prompt_cached_tokens | Prompt tokens served from cache instead of recomputed |
paddock_prefix_cache_queries, paddock_prefix_cache_hits | Prefix-cache effectiveness, which is the number that matters for agent traffic |
paddock_kv_cache_usage_perc, paddock_kv_pages_used, paddock_kv_pages_free | How much of the KV pool is in use, and how close you are to the ceiling |
paddock_num_requests_running, paddock_num_requests_in_flight | Batch occupancy versus everything admitted |
paddock_spec_decode_draft_tokens, paddock_spec_decode_accepted_tokens | Speculative decoding acceptance rate |
paddock_request_failure | Failures by cause |
paddock_build_info | Build identification, as a constant labelled gauge |
Two Formats
The endpoint serves classic Prometheus text by default, which is what most scrapers expect, and OpenMetrics when a client negotiates it through Accept.
The reason to ask for OpenMetrics is exemplars. A histogram bucket can carry the trace identifier of a request that landed in it, which makes a p99 spike something you click through to the exact slow request rather than something you go hunting for. Exemplars cost nothing in cardinality, and classic text format has no way to carry them.
What The Labels Never Carry
No label anywhere carries a session id, a user id, a request id or a key hash. Metrics are metadata about serving, never any part of what was served, and that is enforced by construction rather than by care: label values come from closed sets plus the served model's id.
That is what makes a loopback scrape safe to leave open. A metrics endpoint that could leak conversation identifiers would need protecting; one that structurally cannot does not.
Access And Switching It Off
The default is not "open to everyone". A scrape from loopback needs no key, and a caller arriving over the network does, which is the split that lets a local Prometheus work out of the box without exposing a runner's internals the moment you bind it to an address.
| Flag | Effect |
|---|---|
--metrics-auth on | Require the API key from everyone, loopback included. |
--metrics-auth off | Open to everyone. For a network already closed by a firewall or a sidecar. |
--no-metrics | Remove the endpoint entirely. |
--no-metrics is independent of --no-events, which disables the per-request event ring the activity log is built from. Switching one off leaves the other running.
The same rule bounds cardinality. The label space is operation by origin by the runner's own model ids by HTTP status, all finite. Nothing request-unique can become a label, so the series count cannot grow with traffic.
Benchmarking Tools
This endpoint is also why benchmark numbers are comparable. NVIDIA's aiperf scrapes /metrics every 333 ms during a run by default; before this existed the path returned a JSON 404, aiperf saw content it could not parse and quietly disabled server-metrics collection for the Paddock leg while the vLLM and SGLang legs kept producing theirs. A missing endpoint was silently costing us a column on someone else's scoreboard.
Where Each Number Lives
Metrics come from the process that knows them. A model server reports its own engine counters and latencies on its port. Device-wide GPU telemetry comes from the manager instead, because NVML runs in exactly one process per machine; see Manager API for that feed. A bare runner with no manager therefore reports engine truth and no device temperatures, which is the split a node exporter already covers.
Scrape each runner for engine metrics and the manager for the machine-wide view. There is no federation endpoint that re-exports one through the other, because every fact should have exactly one authoritative producer.