> ## Documentation Index
> Fetch the complete documentation index at: https://polargrid.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Whisper Large V3 Turbo

> Fast multilingual speech-to-text on PolarGrid edge nodes

Whisper Large V3 Turbo (`whisper-large-v3-turbo`) is OpenAI's 809M-parameter Whisper model, served on PolarGrid edge nodes via Triton's `python` backend using the [faster-whisper](https://github.com/SYSTRAN/faster-whisper) runtime. It is co-resident on the voice pod alongside [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) and the `kokoro-82m` TTS model — see [`backend/edge-production-setup/CLAUDE.md`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/CLAUDE.md) for the pod split rationale.

* **HF repo:** [`openai/whisper-large-v3-turbo`](https://huggingface.co/openai/whisper-large-v3-turbo)
* **Modality:** Speech-to-Text (streaming + sync)
* **Backend:** Triton `python` (voice pod)
* **Parameters:** 809M

- **Available regions:** all regions except `dfw-02` — see [Model availability](/guides/model-availability)

<Card title="June 11 update: server-side STT inference halved" icon="gauge-high">
  Re-benched 2026-06-11 on the same node, harness, and input corpus as the
  previous run. Server-only inference dropped **287 → 145 ms p50** (RTF
  **0.050 → 0.025**, \~40× real-time) after the in-process audio resampling fix
  on the STT hot path ([PR #612](https://github.com/PolarGrid-AI/polargrid-monorepo/pull/612)) —
  the gateway previously paid an out-of-process resample on every
  non-16 kHz upload. Streaming time-to-done improved **639 → 582 ms p50**;
  TTFT unchanged at \~78 ms. Raw runs:
  [`benchmarks/yvr-02-2026-06-11/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/benchmarks/yvr-02-2026-06-11).
</Card>

## Headline benchmark

`POST /v1/audio/transcriptions?stream=true` is the live streaming surface. The server emits a `text/event-stream` of `transcript.text.delta` events as each decode window completes, then closes with a single `transcript.text.done` event. Consumers can render the rolling transcript immediately instead of waiting for the final result.

| Measurement | p50 | p95 |
| - | - | - |
| **TTFT (response headers → first non-empty delta)** | **78 ms** | 83 ms |
| Time to `done` event (response headers → done) | 582 ms | 921 ms |
| Interim work (first delta → done) | 504 ms | 841 ms |
| Delta events per request | 4 | 7 (max) |
| Partial deltas (text != final) | 4 | — |
| e2e total (POST → `[DONE]`) | 1304 ms | 1994 ms |
| **RTF (e2e ÷ audio duration)** | **0.23** | 0.30 |
| well\_formed | 100 / 100 | — |

*Bench: 100 streaming transcription runs against `https://api.yvr-02.edge.polargrid.ai`, captured 2026-06-11 from a Vancouver-area laptop. Inputs were the same 5 short utterances (4.2 – 7.7 s, 24 kHz mono WAV, \~200–350 KB each) used by the cohere bench, pre-synthesized via `tada-3b-ml` on the same node. Raw runs: [`benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/whisper-large-v3-turbo_streaming_bench.json).*

> **Delta cadence is chunk-driven.** 4.2 – 5.9 s clips emit 4 deltas, 7.2 – 7.7 s clips emit 7. Deltas are provisional display hints — a delta carries either the new suffix or a full revised hypothesis, and the two are not distinguishable from the event alone, so do not concatenate them; take the final transcript from `done.text` (see [Delta semantics](/api-reference/speech-to-text#delta-semantics)). The first delta arriving at 78 ms after response headers is the meaningful TTFT for live-captioning pipelines. The gateway does not emit `X-Pg-Inference-Ms` on the stream surface yet, so server-only inference cannot be quoted client-side for streaming today.

## Live WebSocket benchmark

`wss://…/v1/audio/transcriptions/ws` transcribes **while audio is being
captured**: the client streams 16 kHz mono PCM frames at microphone pace and
partial transcripts arrive during the utterance — no more waiting for the
upload to finish. Benched 2026-06-11 with the same five clips streamed in
100 ms frames at real-time pace, 100 runs after a 5-run warmup. E2e is
bounded by the audio duration by design; the numbers that matter are the
partial cadence and the end-of-speech lag:

| Metric | p50 | p95 |
| - | - | - |
| First partial (first frame sent → first non-empty delta) | 1023 ms | 1042 ms |
| Partials while audio still flowing | 4 | 7 (max, 7.2–7.7 s clips) |
| **`stop` → authoritative `done`** | **159 ms** | 205 ms |
| well\_formed | 100 / 100 | — |

Raw runs: [`whisper-large-v3-turbo/whisper-large-v3-turbo_ws_bench.json`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/whisper-large-v3-turbo_ws_bench.json).
Harness: [`bench/whisper-large-v3-turbo/bench_ws.py`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/bench/whisper-large-v3-turbo/bench_ws.py).

The first partial lands at the \~1 s window boundary (partials are emitted per
second of new audio) and every run produced partials while audio was still
arriving. The **159 ms p50 stop→done** figure is the effective
end-of-speech-to-transcript latency — the number to compare against
streaming STT providers. The upload surfaces instead pay full-clip
transcription after the audio ends (\~580 ms to done on the SSE surface).
Sessions cap at 120 s of audio; wire protocol and delta semantics in the
[Speech-to-Text API reference](/api-reference/speech-to-text#live-streaming--websocket).

## How this compares

Same node, same inputs, same client as the [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) benches:

| Model | Streaming first-partial p50 | ttft\_done p50 | e2e total p50 | RTF (e2e) |
| - | - | - | - | - |
| `cohere-transcribe-03-2026` | **44 ms** | **560 ms** | 2510 ms | 0.42 |
| **`whisper-large-v3-turbo`** | 77 ms | 639 ms | **1473 ms** | **0.25** |

Cohere reaches its first partial sooner; whisper finishes the full streamed transcript materially faster end-to-end (1473 vs 2510 ms p50) at a lower real-time factor. Pick cohere for earliest-possible interim display and broad accented-language accuracy; pick whisper for fastest full-utterance completion.

*Both rows are from the pre-PR-#612 runs (2026-05-28 / 2026-06-02) so they remain apples-to-apples. Whisper's post-fix numbers are lower (see the headline table above); cohere has not yet been re-benched on the fixed hot path.*

## Quickstart

Edge endpoints accept your raw `pg_*` API key as a bearer token — no token exchange. See [Authentication](/authentication).

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST "https://api.yvr-02.edge.polargrid.ai/v1/audio/transcriptions?sync=true&model=whisper-large-v3-turbo" \
    -H "Authorization: Bearer $POLARGRID_API_KEY" \
    -F "file=@input.wav"
  ```

  ```typescript JavaScript theme={null}
  import { PolarGrid } from "@polargrid/polargrid-sdk";
  import fs from "node:fs";

  const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });

  const result = await client.audioTranscriptions({
    model: "whisper-large-v3-turbo",
    file: fs.createReadStream("input.wav"),
    // Default mode is async (returns job_id). Pass sync to block on the result.
    sync: true,
  });
  console.log(result.text);
  ```

  ```python Python theme={null}
  from polargrid import PolarGrid

  client = await PolarGrid.create(api_key="pg_...")

  with open("input.wav", "rb") as f:
      result = await client.audio_transcriptions({
          "model": "whisper-large-v3-turbo",
          "file": f,
          "sync": True,
      })
  print(result["text"])
  ```
</CodeGroup>

## Endpoint modes

`POST /v1/audio/transcriptions` has three upload modes selected by query params (not multipart fields), plus a live WebSocket surface:

| Mode | Surface | Response | Use when |
| - | - | - | - |
| **Live (WebSocket)** | `wss://…/v1/audio/transcriptions/ws` | `transcript.text.delta` events **while audio is still arriving**, `done` after `stop` | Voice agents and live captioning from a microphone — partials during speech, final transcript \~160 ms after end-of-speech. |
| **Streaming** | `?stream=true` | `text/event-stream` of delta + done events | Rolling display over a complete file upload. |
| **Sync** | `?sync=true` | `200` with the formatted transcript directly | Voice-agent path that needs the answer in one round trip. |
| **Async (default)** | *none* | `202` with `{job_id, poll_url}`; poll via `GET ?job_id=...` | Background batch transcription; no caller blocking. |

`stream` and `sync` are mutually exclusive — passing both returns `400`. Streaming requires `response_format` in `{json, text}`. The WebSocket surface takes 16 kHz mono int16 PCM frames and caps sessions at 120 s — see the [Speech-to-Text API reference](/api-reference/speech-to-text#live-streaming--websocket) for the wire protocol and delta semantics.

### Sync benchmark (`?sync=true`)

The blocking surface holds the connection open until inference completes, then ships the whole JSON response. Client-side TTFB ≈ total wall-clock, and the meaningful split is **server inference time** vs **network leg** (which for STT is dominated by the audio upload). Raw runs: [`benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/whisper-large-v3-turbo_bench.json).

| Measurement | p50 | p95 |
| - | - | - |
| End-to-end TTFB (with network + upload) | 909 ms | 1329 ms |
| **Server-only inference** | **145 ms** | 176 ms |
| *Network leg (e2e − server)* | *765 ms* | — |
| Body transfer (JSON response) | 1.1 ms | 2.4 ms |
| RTF (server inference ÷ audio duration) | 0.025 | 0.031 |

Server-only timing comes from the `X-Pg-Inference-Ms` response header (PR #507), available on the sync surface.

> **The network leg is upload-dominated.** Each request ships a multi-second WAV file before inference can begin, so the 765 ms p50 network figure is largely the upload time of a \~300 KB body — not POP-to-client RTT. For shorter clips (sub-2 s) the network leg shrinks proportionally. Quote the **server-only RTF** when comparing inference throughput against centralized providers.

## Capabilities

| Field | Value |
| - | - |
| Endpoint | `POST /v1/audio/transcriptions` |
| Multipart field | `file` (required) |
| Query params | `model`, `language`, `prompt`, `temperature`, `response_format`, `punctuation`, `stream`, `sync` |
| `response_format` | `json` (default), `text`, `srt`, `vtt`, `verbose_json` |
| Languages | Multilingual — auto-detected, or pin with `language` (e.g. `en`, `fr`, `es`) |
| Max batch size | 1 |
| Backend pod | `inference-backend-triton-voice` |

## Response timing headers

PR #507 added two response headers that bench harnesses and observability tooling can read to get a server-only inference time without inferring it from the body:

| Header | Value |
| - | - |
| `X-Pg-Inference-Ms` | Integer milliseconds — wall-clock around the `transcribe()` call inside the gateway's `_handle_sync` (i.e., excludes the audio upload + response transfer). |
| `Server-Timing` | Standard-shape `inference;dur=<ms>` entry carrying the same number. |

These let callers compute the network leg as `(client wall-clock) − (X-Pg-Inference-Ms)`, the same e2e-vs-server split available for LLM via `pg_metadata`.

## Model identifier

Call this model with the canonical id `whisper-large-v3-turbo` at `/v1/audio/transcriptions`. It has no short alias. The HuggingFace repo id `openai/whisper-large-v3-turbo` is accepted at `/v1/models/load` for hot-loading purposes but does not resolve at inference time.

## Notes

* License: [Apache 2.0](https://huggingface.co/openai/whisper-large-v3-turbo) (no auth required to pull weights).
* Runtime: served via `faster-whisper`, not raw `transformers`, for optimized CTranslate2 inference.
* For multilingual coverage and accuracy on accented speech, [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) is the alternative — whisper-turbo's edge is lowest full-utterance latency.

## See also

* [Speech-to-Text API](/api-reference/speech-to-text) — endpoint reference, formats, streaming contract
* [Voice AI guide](/guides/voice) — building voice agents on PolarGrid
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.