Speech to Text

Paddock serves speech models on two surfaces: one request for a file you already have, and a WebSocket for audio that is still being spoken. Both follow OpenAI's wire format, so existing clients and SDKs work unchanged.

Transcribing A File

POST /v1/audio/transcriptions takes multipart/form-data.

Parameters in the OpenAI specification that Paddock does not serve are refused by name rather than ignored, and a field that is not in the specification at all is refused the same way the JSON endpoints refuse an unknown key. You never get a quietly different answer than the one you asked for.

Streaming A File

With stream=true the response is a sequence of transcript.text.delta events followed by one transcript.text.done. The deltas concatenate to exactly the text the terminal event carries, so a client that appends every delta ends up holding the final transcript with no reconciliation step.

srt and vtt refuse to stream. Half a subtitle document is not half an answer.

Timestamps

Timestamps are opt-in through timestamp_granularities[], and which of these a checkpoint can answer is a property of that checkpoint. The server publishes it on GET /v1/models, so a client can read the capability instead of discovering it through a 400. Asking a model for a granularity it cannot produce returns a refusal naming the model, never an empty segments array.

On Whisper, segment changes the decode: the model is told to emit times and its output is constrained by the timestamp grammar. That is why it is opt-in, since a plain transcription keeps the exact prompt the accuracy gates were measured against. Asking for word adds a second, teacher-forced alignment pass over each window, which is where OpenAI's note about word timestamps costing extra latency comes from; it does not change the transcript. Both compose, giving a top-level words[] of {word, start, end} alongside segments[].

On Granite Speech Plus, word timing is a different mechanism: the model is asked for the times and writes them into its own answer, so here the granularity does change the transcript. IBM's model card is explicit that the timestamp mode drops punctuation and capitalization, which is also why segment is refused on that family, since without punctuation there are no sentence boundaries to cut cues on.

Live Transcription

GET /v1/realtime?intent=transcription upgrades to a WebSocket carrying OpenAI's Realtime transcription session. A microphone has no end until the speaker stops, so it cannot be a request body; this is the shape OpenAI defined for it and clients already handle it.

What Live Text Guarantees

Whisper's encoder reads a fixed 30-second window, so something has to decide which words are safe to show before the sentence is finished. Paddock commits only the longest prefix that two consecutive passes agree on, a policy known as LocalAgreement-2.

Words already shown are never retracted, and the deltas concatenate to the completed transcript exactly as they do on the file endpoint. What follows from that: completed is not the same text you would get by transcribing the whole recording in one pass. It is the committed prefix plus the final pass's tail, and that prefix was decided from shorter buffers. Not rewriting what you already read is what a delta stream buys, and this is what it costs.

A closed utterance runs a final, authoritative pass, and completed carries a paddock_verbose object holding exactly what verbose_json would return for that utterance's audio, including segment times, word times and per-word confidence. Cost therefore scales with speech rather than with recording length: a long session settles as it goes instead of being re-transcribed at the end.

Aligning A Transcript You Already Have

POST /v1/audio/alignments answers a different question from transcription: given the audio and the words, when was each one said. That is what you need to caption an existing script, to check a proofread transcript against the recording, or to jump to the moment a phrase occurs.

Multipart form: file for the audio, text for the transcript to align, and optionally language. The reply's words array is the same {word, start, end} shape in seconds that verbose_json uses for transcription word times, so anything that already reads those needs no new parser.

OpenAI has no alignment endpoint, so this one is Paddock's own. It borrows the word shape and nothing else, which is the only part worth borrowing.

Its Two Honest Limits

The model addresses time in fixed classes over a bounded budget, which caps the clip it can align at roughly 6.7 minutes on the 0.6B. A longer clip is refused with the cap stated in the message rather than truncated to fit.

Japanese and Korean are refused outright. Aligning them properly needs tokenizers that understand word structure, which this build does not carry, and the default splitter would hand back clause-sized "words" that look like an answer and are not one. Languages outside the model's trained eleven are accepted and flagged: the response carries language_supported: false, because the alignment may still be useful and you should be the one deciding that.

Who Spoke When

Diarization is a separate question from transcription, and a separate model: it marks out speaker activity over time and writes no words. POST /v1/audio/diarizations takes the audio as multipart, with an optional preset and an optional words array.

curl -X POST localhost:11540/v1/audio/diarizations \
  -F 'file=@meeting.wav' -F 'preset=offline'

{
  "model": "nemotron-3-diarization",
  "duration": 94.2,
  "frame_seconds": 0.01,
  "num_speakers": 3,
  "segments": [{"speaker": 0, "start": 0.4, "end": 6.1}, ...],
  "words": null,
  "attribution": "time_overlap_v1"
}

Up to eight speakers, at 10 ms resolution, including people talking over each other. Recordings up to ten minutes, 16 kHz mono.

Attaching A Transcript

Pass a words array with timings and each word comes back with the speaker whose segment it overlaps, under the time_overlap_v1 rule. That is how you get an attributed transcript: transcribe with one model, diarize with this one, and join them on time.

Live Audio

GET /v1/audio/diarizations/stream is a WebSocket taking 16 kHz mono pcm_s16le in chunks of up to 16,000 samples, and reporting speaker activity as it goes. Four latency presets trade delay against accuracy:

The Studio shows the result as a speaker timeline beside the transcript, with each word attributed, kept with the conversation.

Loading On Demand

A Whisper endpoint usually holds its model in GPU memory from the moment the server starts, which is the right trade when transcriptions arrive all day and the wrong one when they arrive twice a week. Set the endpoint to load on first request instead, and it stays up holding no GPU memory, loads the model when a transcription arrives, and unloads it again after an idle period you choose.

Two settings under Model loading in the endpoint's settings:

The change applies without restarting the server. The first request after an idle period waits for the load, so it is slower than the ones that follow; the load has its own timeout, two minutes by default, after which the request fails rather than hanging.

Requests that arrive while the model is still loading queue and are served once it is up, rather than each triggering a load of their own.

Models

Six speech models ship in the catalog: Whisper fine-tunes for Swedish, Norwegian and Danish, plus the multilingual Qwen3-ASR and Granite Speech families. See Supported Models for the list and what each one is good at.