# Chat Completions
Source: https://polargrid.mintlify.app/api-reference/chat-completions
Generate chat completions with conversation context
# Chat Completions
The chat completions endpoint is the recommended way to generate text. It supports multi-turn conversations with system, user, and assistant messages.
Edge endpoints require a JWT. See [Authentication](/authentication) for how to obtain one.
## Create Chat Completion
```
POST /v1/chat/completions
```
Generate a chat completion for the given messages.
### Request Body
| Parameter | Type | Required | Default | Description |
| ------------------- | ------- | -------- | ------- | ----------------------------------- |
| `model` | string | Yes | — | Model ID (e.g., `qwen-3.5-9b`) |
| `messages` | array | Yes | — | Array of message objects |
| `max_tokens` | integer | No | 150 | Maximum tokens to generate (1-4096) |
| `temperature` | number | No | 0.7 | Sampling temperature (0.0-2.0) |
| `top_p` | number | No | 0.9 | Nucleus sampling (0.0-1.0) |
| `top_k` | integer | No | — | Top-k sampling |
| `frequency_penalty` | number | No | 0.0 | Frequency penalty (-2.0 to 2.0) |
| `presence_penalty` | number | No | 0.0 | Presence penalty (-2.0 to 2.0) |
| `stop` | array | No | — | Up to 4 stop sequences |
| `stream` | boolean | No | false | Enable streaming |
| `user` | string | No | — | End-user identifier |
**Function / tool calling is not supported.** The `tools`, `functions`, and `function_call` parameters are silently ignored by the chat completions endpoint. To use tools, parse the model's text output and dispatch tool calls in your application code.
### Message Format
```json theme={null}
{
"role": "system" | "user" | "assistant",
"content": "message text"
}
```
### Example Request
```bash cURL theme={null}
# Edge endpoints require a JWT — see Authentication
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.5-9b",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
],
"max_tokens": 100,
"temperature": 0.7
}'
```
```javascript JavaScript theme={null}
const response = await client.chatCompletion({
model: 'qwen-3.5-9b',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is the capital of France?' }
],
maxTokens: 100,
temperature: 0.7,
});
console.log(response.choices[0].message.content);
```
```python Python theme={null}
response = await client.chat_completion({
"model": "qwen-3.5-9b",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
],
"max_tokens": 100,
"temperature": 0.7,
})
print(response.choices[0].message.content)
```
### Response
```json theme={null}
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1234567890,
"model": "qwen-3.5-9b",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The capital of France is Paris."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 8,
"total_tokens": 33
}
}
```
## Streaming
For real-time responses, enable streaming:
```javascript JavaScript theme={null}
for await (const chunk of client.chatCompletionStream({
model: 'qwen-3.5-9b',
messages: [{ role: 'user', content: 'Tell me a story' }],
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) {
process.stdout.write(content);
}
}
```
```python Python theme={null}
async for chunk in client.chat_completion_stream({
"model": "qwen-3.5-9b",
"messages": [{"role": "user", "content": "Tell me a story"}],
}):
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
### Stream Response Format
Each chunk is a Server-Sent Event:
```
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":"The"}}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":" capital"}}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
## Finish Reasons
| Reason | Description |
| ---------------- | --------------------------------------- |
| `stop` | Natural completion or stop sequence hit |
| `length` | Max tokens reached |
| `content_filter` | Content filtered |
# Completions
Source: https://polargrid.mintlify.app/api-reference/completions
Generate text completions from a prompt
# Completions
The completions endpoint generates text from a single prompt. For conversational use cases, prefer [Chat Completions](/api-reference/chat-completions).
Edge endpoints accept your `pg_*` API key as a bearer token. See [Authentication](/authentication) for details. The cURL examples below pin Toronto (`yto-01`) for concreteness — substitute another region or discover the fastest one via `GET https://autorouter.polargrid.ai/v1/route`. See [API Overview](/api-reference/overview#picking-a-region) for both patterns.
## Create Completion
```
POST /v1/completions
```
Generate a completion for the given prompt.
### Request Body
| Parameter | Type | Required | Default | Description |
| ------------------- | ------- | -------- | ------- | ------------------------------- |
| `prompt` | string | Yes | — | The prompt to complete |
| `model` | string | Yes | — | Model ID |
| `max_tokens` | integer | No | 100 | Maximum tokens (1-4096) |
| `temperature` | number | No | 0.7 | Sampling temperature (0.0-2.0) |
| `top_p` | number | No | 0.9 | Nucleus sampling (0.0-1.0) |
| `top_k` | integer | No | 50 | Top-k sampling |
| `frequency_penalty` | number | No | 0.0 | Frequency penalty (-2.0 to 2.0) |
| `presence_penalty` | number | No | 0.0 | Presence penalty (-2.0 to 2.0) |
| `stop` | array | No | — | Up to 4 stop sequences |
| `user` | string | No | — | End-user identifier |
### Example Request
```bash cURL theme={null}
# Edge endpoints accept your pg_* API key as a bearer token — see Authentication
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/completions \
-H "Authorization: Bearer pg_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"prompt": "Once upon a time",
"model": "qwen-3.8-27b",
"max_tokens": 100,
"temperature": 0.8
}'
```
```javascript JavaScript theme={null}
const response = await client.completion({
prompt: 'Once upon a time',
model: 'qwen-3.8-27b',
maxTokens: 100,
temperature: 0.8,
});
console.log(response.choices[0].text);
```
```python Python theme={null}
response = await client.completion({
"prompt": "Once upon a time",
"model": "qwen-3.8-27b",
"max_tokens": 100,
"temperature": 0.8,
})
print(response.choices[0].text)
```
### Response
```json theme={null}
{
"id": "cmpl-abc123",
"object": "text_completion",
"created": 1234567890,
"model": "qwen-3.8-27b",
"choices": [
{
"text": " in a land far away, there lived a young princess...",
"index": 0,
"logprobs": null,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 4,
"completion_tokens": 100,
"total_tokens": 104
}
}
```
## Streaming
Enable streaming for real-time token generation:
```javascript JavaScript theme={null}
for await (const chunk of client.completionStream({
prompt: 'Once upon a time',
model: 'qwen-3.8-27b',
})) {
process.stdout.write(chunk.choices[0].text);
}
```
```python Python theme={null}
async for chunk in client.completion_stream({
"prompt": "Once upon a time",
"model": "qwen-3.8-27b",
}):
print(chunk.choices[0].text, end="", flush=True)
```
## Legacy Generate Method
The SDKs also provide a `generate()` method for backward compatibility. It wraps `chatCompletion()` internally:
```javascript theme={null}
const response = await client.generate({
model: 'qwen-3.8-27b',
prompt: 'Hello, how are you?',
maxTokens: 100,
});
console.log(response.content);
console.log(`Processing time: ${response.processingTimeMs}ms`);
```
# GPU
Source: https://polargrid.mintlify.app/api-reference/gpu
Monitor and manage GPU resources
# GPU
Monitor GPU utilization and manage GPU memory on edge nodes.
Edge endpoints accept your `pg_*` API key as a bearer token. See [Authentication](/authentication) for details. The cURL examples below pin Toronto (`yto-01`) for concreteness — substitute another region or discover the fastest one via `GET https://autorouter.polargrid.ai/v1/route`. See [API Overview](/api-reference/overview#picking-a-region) for both patterns.
**Operator-only.** `/v1/gpu/*` requires a `superadmin`-scoped credential, issued only to PolarGrid operators. Standard `pg_*` API keys receive **403 Forbidden**. These endpoints act on node-global, multi-tenant GPU state — they are documented for operator reference, not customer inference traffic.
## Get GPU Status
```
GET /v1/gpu/status
```
Get detailed GPU status including memory, utilization, and temperature.
### Example
```bash cURL theme={null}
# Edge endpoints accept your pg_* API key as a bearer token — see Authentication
curl https://api.yto-01.edge.polargrid.ai/v1/gpu/status \
-H "Authorization: Bearer pg_your_api_key"
```
```javascript JavaScript theme={null}
const gpuStatus = await client.getGPUStatus();
gpuStatus.gpus.forEach(gpu => {
console.log(`GPU ${gpu.index}: ${gpu.name}`);
console.log(` Memory: ${gpu.memory.usedGb}GB / ${gpu.memory.totalGb}GB`);
console.log(` Utilization: ${gpu.utilization.gpuPercent}%`);
console.log(` Temperature: ${gpu.temperatureC}°C`);
});
```
```python Python theme={null}
gpu_status = await client.get_gpu_status()
for gpu in gpu_status.gpus:
print(f"GPU {gpu.index}: {gpu.name}")
print(f" Memory: {gpu.memory.used_gb}GB / {gpu.memory.total_gb}GB")
print(f" Utilization: {gpu.utilization.gpu_percent}%")
print(f" Temperature: {gpu.temperature_c}°C")
```
### Response
```json theme={null}
{
"status": "success",
"timestamp": "2025-01-29T12:00:00Z",
"gpus": [
{
"index": 0,
"name": "NVIDIA A100 80GB",
"memory": {
"total_mb": 81920,
"used_mb": 45056,
"free_mb": 36864,
"total_gb": 80.0,
"used_gb": 44.0,
"free_gb": 36.0,
"percent_used": 55.0
},
"utilization": {
"gpu_percent": 72,
"memory_percent": 55
},
"temperature_c": 65
}
],
"processes": [
{
"pid": 12345,
"name": "python",
"memory_mb": 40960
}
],
"total_gpus": 1
}
```
## Get GPU Memory
```
GET /v1/gpu/memory
```
Get simplified GPU memory information.
### Example
```javascript JavaScript theme={null}
const memory = await client.getGPUMemory();
memory.memory.forEach((gpu, i) => {
console.log(`GPU ${i}: ${gpu.usedGb}GB / ${gpu.totalGb}GB (${gpu.percentUsed}%)`);
});
```
```python Python theme={null}
memory = await client.get_gpu_memory()
for i, gpu in enumerate(memory.memory):
print(f"GPU {i}: {gpu.used_gb}GB / {gpu.total_gb}GB ({gpu.percent_used}%)")
```
### Response
```json theme={null}
{
"status": "success",
"timestamp": "2025-01-29T12:00:00Z",
"memory": [
{
"total_gb": 80.0,
"used_gb": 44.0,
"free_gb": 36.0,
"percent_used": 55.0,
"percent_free": 45.0
}
]
}
```
## Purge GPU Memory
```
POST /v1/gpu/purge
```
Unload all models and clear GPU memory cache.
### Request Body
| Parameter | Type | Required | Default | Description |
| --------- | ------- | -------- | ------- | ------------------------------------- |
| `force` | boolean | No | false | Force purge even if models are in use |
### Example
```javascript JavaScript theme={null}
const result = await client.purgeGPU({ force: false });
console.log(`Freed ${result.memoryFreedGb}GB`);
console.log(`Unloaded models:`, result.modelsUnloaded);
console.log(result.recommendation);
```
```python Python theme={null}
result = await client.purge_gpu({"force": False})
print(f"Freed {result.memory_freed_gb}GB")
print(f"Unloaded models: {result.models_unloaded}")
print(result.recommendation)
```
### Response
```json theme={null}
{
"status": "success",
"timestamp": "2025-01-29T12:00:00Z",
"actions": ["unloaded qwen-3.8-27b", "cleared CUDA cache"],
"memory_before": {
"used_gb": 44.0,
"total_gb": 80.0,
"percent_used": 55.0
},
"memory_after": {
"used_gb": 2.1,
"total_gb": 80.0,
"percent_used": 2.6
},
"memory_freed_gb": 41.9,
"models_unloaded": ["qwen-3.8-27b"],
"errors": [],
"recommendation": "GPU memory cleared successfully"
}
```
# Health
Source: https://polargrid.mintlify.app/api-reference/health
Check service health status
# Health
Check the health status of PolarGrid edge services.
## Health Check
```
GET /health
```
Check service health and available features.
### Example
```bash cURL theme={null}
# /health is unauthenticated — no Bearer token needed
curl https://api.yto-01.edge.polargrid.ai/health
```
```javascript JavaScript theme={null}
const health = await client.health();
console.log(`Status: ${health.status}`);
console.log(`Backend healthy: ${health.backend.healthy}`);
```
```python Python theme={null}
health = await client.health()
print(f"Status: {health.status}")
print(f"Backend healthy: {health.backend.healthy}")
```
### Response
```json theme={null}
{
"status": "healthy",
"node": "pg-edge-tor-01",
"runtime": "kata",
"timestamp": "2026-05-11T16:37:29.110635",
"features": {
"dynamic_loading": true,
"huggingface_support": true
},
"backend": {
"healthy": true,
"info": {
"name": "triton",
"version": "2.67.0",
"extensions": [
"classification", "sequence", "model_repository",
"model_repository(unload_dependents)", "schedule_policy",
"model_configuration", "system_shared_memory",
"cuda_shared_memory", "binary_tensor_data",
"parameters", "statistics", "trace", "logging"
]
}
}
}
```
### Status Values
| Status | Description |
| ----------- | ------------------------------- |
| `healthy` | All systems operational |
| `degraded` | Partial functionality available |
| `unhealthy` | Service unavailable |
### Features
| Feature | Description |
| --------------------- | -------------------------------------------------------- |
| `dynamic_loading` | Gateway supports loading and unloading models at runtime |
| `huggingface_support` | Gateway can pull models from the HuggingFace registry |
## CLI Health Check
You can also check health via the CLI:
```bash theme={null}
# Check all regions
polargrid test health
# Check specific region
polargrid test health --region yto-01
```
# Model Loading
Source: https://polargrid.mintlify.app/api-reference/model-loading
Manage models on edge nodes (internal infrastructure)
# Model Loading
PolarGrid supports dynamic model loading for managing models on edge nodes.
**Operator-only.** `/v1/models/load`, `/v1/models/unload`, and `/v1/models/unload-all` require a `superadmin`-scoped credential, issued only to PolarGrid operators. Standard `pg_*` API keys receive **403 Forbidden**. Models available for inference are pre-deployed across edge regions — you do not need to load models yourself. Use `GET /v1/models` to see which models are available in your region.
Edge endpoints accept your `pg_*` API key as a bearer token. See [Authentication](/authentication) for details. The cURL examples below pin Toronto (`yto-01`) for concreteness — substitute another region or discover the fastest one via `GET https://autorouter.polargrid.ai/v1/route`. See [API Overview](/api-reference/overview#picking-a-region) for both patterns.
## Load Model
```
POST /v1/models/load
```
Load a model into GPU memory.
### Request Body
| Parameter | Type | Required | Default | Description |
| -------------- | ------- | -------- | ------- | ----------------------------------- |
| `model_name` | string | Yes | — | Model ID to load |
| `force_reload` | boolean | No | false | Force reload even if already loaded |
### Example
```bash cURL theme={null}
# Edge endpoints accept your pg_* API key as a bearer token — see Authentication
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/models/load \
-H "Authorization: Bearer pg_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"model_name": "qwen-3.8-27b",
"force_reload": false
}'
```
```javascript JavaScript theme={null}
const result = await client.loadModel({
modelName: 'qwen-3.8-27b',
forceReload: false,
});
console.log(result.message); // "Model qwen-3.8-27b loaded successfully"
```
```python Python theme={null}
result = await client.load_model({
"model_name": "qwen-3.8-27b",
"force_reload": False,
})
print(result.message)
```
### Response
```json theme={null}
{
"status": "success",
"model": "qwen-3.8-27b",
"force_reload": false,
"message": "Model qwen-3.8-27b loaded successfully"
}
```
## Unload Model
```
POST /v1/models/unload
```
Unload a model from GPU memory.
### Request Body
| Parameter | Type | Required | Description |
| ------------ | ------ | -------- | ----------------------------------------- |
| `model_name` | string | Yes | Model ID to unload (e.g., `qwen-3.8-27b`) |
### Example
```javascript JavaScript theme={null}
const result = await client.unloadModel({
modelName: 'gpt2',
});
console.log(result.message);
```
```python Python theme={null}
result = await client.unload_model({
"model_name": "gpt2",
})
print(result.message)
```
### Response
```json theme={null}
{
"status": "success",
"model": "gpt2",
"message": "Model gpt2 unloaded successfully"
}
```
## Unload All Models
```
POST /v1/models/unload-all
```
Unload all models from GPU memory.
### Example
```javascript JavaScript theme={null}
const result = await client.unloadAllModels();
console.log(`Unloaded ${result.totalUnloaded} models`);
console.log('Models:', result.unloadedModels);
```
```python Python theme={null}
result = await client.unload_all_models()
print(f"Unloaded {result.total_unloaded} models")
print("Models:", result.unloaded_models)
```
### Response
```json theme={null}
{
"status": "success",
"unloaded_models": ["qwen-3.8-27b", "whisper-large-v3-turbo"],
"errors": [],
"total_unloaded": 2
}
```
## Get Model Status
```
GET /v1/models/status
```
Get the loading status of all models.
### Example
```javascript JavaScript theme={null}
const status = await client.getModelStatus();
console.log('Loaded models:', status.loaded);
console.log('Status:', status.loadingStatus);
```
```python Python theme={null}
status = await client.get_model_status()
print("Loaded models:", status.loaded)
print("Status:", status.loading_status)
```
### Response
```json theme={null}
{
"loaded": ["qwen-3.8-27b", "whisper-large-v3-turbo"],
"loading_status": {
"qwen-3.8-27b": "loaded",
"whisper-large-v3-turbo": "loaded",
"gpt2": "unloaded"
},
"repository": "/models"
}
```
## Status Values
| Status | Description |
| ---------- | -------------------------------- |
| `loaded` | Model is in GPU memory and ready |
| `loading` | Model is currently being loaded |
| `unloaded` | Model is not in memory |
| `failed` | Model failed to load |
# Models
Source: https://polargrid.mintlify.app/api-reference/models
List and query available models
# Models
List available models on the edge infrastructure.
Edge endpoints accept your `pg_*` API key as a bearer token. See [Authentication](/authentication) for details. The cURL examples below pin Toronto (`yto-01`) for concreteness — substitute another region or discover the fastest one via `GET https://autorouter.polargrid.ai/v1/route`. See [API Overview](/api-reference/overview#picking-a-region) for both patterns.
## List Models
```
GET /v1/models
```
List all available models.
### Example Request
```bash cURL theme={null}
# Edge endpoints accept your pg_* API key as a bearer token — see Authentication
curl https://api.yto-01.edge.polargrid.ai/v1/models \
-H "Authorization: Bearer pg_your_api_key"
```
```javascript JavaScript theme={null}
const { data: models } = await client.listModels();
models.forEach(model => {
console.log(`${model.id} (${model.ownedBy})`);
});
```
```python Python theme={null}
response = await client.list_models()
for model in response.data:
print(f"{model.id} ({model.owned_by})")
```
### Response
```json theme={null}
{
"object": "list",
"data": [
{
"id": "qwen-3.8-27b",
"object": "model",
"created": 1234567890,
"owned_by": "qwen",
"permission": [],
"root": "qwen-3.8-27b",
"parent": null
},
{
"id": "qwen-3.8-27b",
"object": "model",
"created": 1234567890,
"owned_by": "qwen",
"permission": [],
"root": "qwen-3.8-27b",
"parent": null
},
{
"id": "kokoro-82m",
"object": "model",
"created": 1234567890,
"owned_by": "kokoro",
"permission": [],
"root": "kokoro-82m",
"parent": null
},
{
"id": "whisper-large-v3-turbo",
"object": "model",
"created": 1234567890,
"owned_by": "openai",
"permission": [],
"root": "whisper-large-v3-turbo",
"parent": null
},
{
"id": "cohere-transcribe-03-2026",
"object": "model",
"created": 1234567890,
"owned_by": "cohere",
"permission": [],
"root": "cohere-transcribe-03-2026",
"parent": null
},
{
"id": "tada-3b-ml",
"object": "model",
"created": 1234567890,
"owned_by": "hume",
"permission": [],
"root": "tada-3b-ml",
"parent": null
}
]
}
```
## Available Model Types
### Text Generation
| Model | Size | Use Case |
| -------------- | ---- | ------------------------------------------------- |
| `qwen-3.8-27b` | 27B | High quality, complex reasoning, tools, JSON mode |
### Speech-to-Text
| Model | Description |
| --------------------------- | --------------------------------- |
| `whisper-large-v3-turbo` | Fast, accurate |
| `cohere-transcribe-03-2026` | Cohere multilingual transcription |
### Text-to-Speech
| Model | Description |
| ------------ | ------------------- |
| `kokoro-82m` | Lightweight, fast |
| `tada-3b-ml` | Hume expressive TTS |
# API Overview
Source: https://polargrid.mintlify.app/api-reference/overview
OpenAI-compatible REST API for edge AI inference
# API Overview
PolarGrid provides an OpenAI-compatible REST API, making it easy to migrate existing applications or use familiar patterns.
## Base URL
All inference traffic targets a regional edge gateway directly:
```
https://api.{region}.edge.polargrid.ai
```
Available regions (16):
* `yto-01` — Toronto
* `yul-01` — Montreal
* `yvr-02` — Vancouver
* `nyc-01` — New York
* `nyc-02` — New York 02
* `dfw-01` — Dallas
* `dfw-02` — Dallas 02
* `sfo-01` — San Francisco
* `lax-01` — Los Angeles
* `sea-01` — Seattle
* `chi-01` — Chicago
* `phx-01` — Phoenix
* `was-01` — Washington DC
* `mia-01` — Miami
* `sfo-03` — San Francisco
### Picking a region
There are two patterns:
1. **Auto-route via the autorouter** (recommended). Make a single `GET https://autorouter.polargrid.ai/v1/route` to discover the best edge for the caller — by default the nearest edge **in the caller's country** (falling back to a network-aware cross-border pick if that country has no edge) — then use the returned `endpoint` as the base URL for all subsequent requests. Pass `?scope=global` to route to the nearest edge worldwide instead, ignoring borders. This is what the SDKs do internally — see [Regions](/guides/regions).
2. **Pin a specific region.** Skip the autorouter and hit `https://api.{region}.edge.polargrid.ai` directly when you need predictable routing.
`autorouter.polargrid.ai` is a discovery service, not an inference proxy. It serves `GET /v1/route` only. Sending `POST /v1/chat/completions` (or any other inference verb) to the autorouter returns 403 — CloudFront on that distribution only allows cacheable requests.
## Authentication
Edge endpoints accept your `pg_*` API key directly — attach it as a bearer token on every request. See [Authentication](/authentication) for full details.
```bash theme={null}
# Send the API key directly to any edge endpoint.
curl -s "https://api.yto-01.edge.polargrid.ai/v1/models" \
-H "Authorization: Bearer pg_your_api_key"
```
Get your API key from the Console.
## Endpoints
### Text Inference
| Method | Endpoint | Description |
| ------ | ---------------------- | ------------------------------ |
| POST | `/v1/chat/completions` | Chat completions (recommended) |
| POST | `/v1/completions` | Text completions |
### Audio
| Method | Endpoint | Description |
| ------ | -------------------------- | -------------- |
| POST | `/v1/audio/speech` | Text-to-speech |
| POST | `/v1/audio/transcriptions` | Speech-to-text |
### Models
| Method | Endpoint | Description |
| ------ | ------------------- | ------------------------ |
| GET | `/v1/models` | List available models |
| GET | `/v1/models/status` | Get model loading status |
### Operator endpoints (superadmin only)
The model-lifecycle and GPU endpoints below act on node-global, multi-tenant state. They require a `superadmin`-scoped credential issued only to PolarGrid operators — standard `pg_*` API keys receive **403 Forbidden**. You never need these for inference; models are pre-deployed per region (see [Model Loading](/api-reference/model-loading) and [GPU](/api-reference/gpu)).
| Method | Endpoint | Description |
| ------ | ----------------------- | ---------------------------- |
| POST | `/v1/models/load` | Load a model into GPU memory |
| POST | `/v1/models/unload` | Unload a model |
| POST | `/v1/models/unload-all` | Unload all models |
| GET | `/v1/gpu/status` | Detailed GPU status |
| GET | `/v1/gpu/memory` | GPU memory usage |
| POST | `/v1/gpu/purge` | Clear GPU memory |
### Health
| Method | Endpoint | Description |
| ------ | --------- | -------------------- |
| GET | `/health` | Service health check |
## Request Format
Most POST requests accept JSON. The exception is speech-to-text (`POST /v1/audio/transcriptions`), which takes a `multipart/form-data` file upload with its options as query parameters — see [Speech to Text](/api-reference/speech-to-text).
```bash theme={null}
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer pg_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello"}],
"max_tokens": 100
}'
```
## Response Format
Responses are JSON with this structure:
```json theme={null}
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1234567890,
"model": "qwen-3.8-27b",
"choices": [...],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 20,
"total_tokens": 30
}
}
```
## Errors
Errors return appropriate HTTP status codes with a detail string (FastAPI format):
```json theme={null}
{
"detail": "Invalid API key"
}
```
| Status | Description |
| ------ | ------------------------------ |
| 400 | Bad request (validation error) |
| 401 | Unauthorized (invalid API key) |
| 404 | Not found |
| 429 | Rate limit exceeded |
| 500 | Server error |
## Streaming
For streaming responses, set `stream: true`:
```bash theme={null}
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer pg_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Tell me a story"}],
"stream": true
}'
```
Streaming responses use Server-Sent Events (SSE). See the [Streaming Guide](/guides/streaming) for details.
## OpenAI Compatibility
PolarGrid's REST API follows OpenAI's endpoint structure and request/response formats, so tools that speak the OpenAI wire protocol (e.g., `curl`, LangChain, LiteLLM) can target PolarGrid with a base URL change.
The PolarGrid Python and JavaScript SDKs use their own method signatures (e.g., `client.chat_completion({...})` instead of `client.chat.completions.create(...)`) and are **not** drop-in replacements for the OpenAI SDK. See the [SDK docs](/sdks/overview) for details.
**Known limitations vs. OpenAI:**
| Feature | Status | Notes |
| ---------------------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Tool/function calling (`tools`, `tool_choice`, `functions`, `function_call`) | Limited | The gateway processes tools and parses `tool_calls` from model output, but the currently deployed model (`qwen-3.8-27b`) has limited tool-calling reliability. See [Chat Completions](/api-reference/chat-completions). |
| Image / vision input (`image_url` content parts) | Limited | Supported on vision-capable models on vision-enabled nodes — currently `qwen-3.6-35b-a3b` on the `sfo-02` staging node. Text-only models return `400` for image content. See [Chat Completions → Vision](/api-reference/chat-completions#vision-image-input). |
| `response_format` default for TTS | Differs | REST defaults to `pcm`; OpenAI defaults to `mp3`. The PolarGrid SDKs default to `mp3` (batch) / `opus` (streaming) for parity. Pass `response_format` explicitly to avoid surprises. |
| Streaming TTS formats | Partial | Only `pcm` and `opus` are streamable. `wav` and `mp3` are batch-only. |
# Speech-to-Text
Source: https://polargrid.mintlify.app/api-reference/speech-to-text
Transcribe audio to text
# Speech-to-Text
Transcribe audio files to text. A single endpoint serves three modes — default async, opt-in streaming, opt-in sync — controlled by **query parameters**.
Edge endpoints accept your `pg_*` API key as a bearer token. See [Authentication](/authentication) for details. The cURL examples below pin Toronto (`yto-01`) for concreteness — substitute another region or discover the fastest one via `GET https://autorouter.polargrid.ai/v1/route`.
## Transcribe Audio
```
POST /v1/audio/transcriptions
```
The multipart body carries **only the `file`**. Everything else is a query parameter.
### Query Parameters
| Parameter | Type | Required | Default | Description |
| ----------------- | ------- | -------- | ------- | ---------------------------------------------------------------------- |
| `model` | string | Yes | — | STT model (e.g. `whisper-large-v3-turbo`, `cohere-transcribe-03-2026`) |
| `language` | string | No | — | ISO-639-1 language code |
| `prompt` | string | No | — | Context hint to guide transcription |
| `response_format` | string | No | `json` | `json`, `text`, `srt`, `vtt`, or `verbose_json` |
| `temperature` | number | No | 0 | Sampling temperature (0.0-1.0) |
| `punctuation` | boolean | No | — | Force/forbid punctuation in the output |
| `stream` | boolean | No | `false` | If `true`, return Server-Sent Events |
| `sync` | boolean | No | `false` | If `true`, block until completion and return the result inline |
### Three Modes
| Query flags | Response | Use when |
| -------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------- |
| *(none)* | **`202 Accepted`** with `{ job_id, status, poll_url }` | **Default.** Best for long files; poll with `GET /v1/audio/transcriptions?job_id=...` |
| `?stream=true` | `200` **SSE** of `transcript.text.delta` + `transcript.text.done` | Best UX for real-time display |
| `?sync=true` | `200` with formatted body (JSON / text / SRT / VTT / verbose JSON) | Small files when you can wait inline |
`stream=true` and `sync=true` are mutually exclusive.
### OpenAI SDK Drop-In
OpenAI's transcription endpoint is strictly synchronous, and stock OpenAI
SDKs send `model`, `language`, `response_format`, and `temperature` as
**multipart form fields** — they have no way to add query parameters. The
endpoint accepts both shapes:
* **Form fields are honored as fallbacks** for any query parameter you
don't set (query always wins when both are present).
* **A request whose `model` arrives as a form field with no mode flags is
served synchronously** — a stock OpenAI SDK gets the transcript inline,
exactly as it does against OpenAI.
```python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="pg_your_api_key",
base_url="https://api.yto-01.edge.polargrid.ai/v1",
)
result = client.audio.transcriptions.create(
model="whisper-large-v3-turbo",
file=open("recording.mp3", "rb"),
)
print(result.text)
```
The async-job default (`202` + `poll_url`) applies only to PolarGrid-style
requests that pass `model` as a query parameter. If you build raw requests
and want inline results, pass `?sync=true`.
### Available Models
| Model | Description |
| --------------------------- | ----------------------------------------------- |
| `whisper-large-v3-turbo` | OpenAI Whisper, fast multilingual transcription |
| `cohere-transcribe-03-2026` | Cohere transcription, 14 languages |
### Examples
#### Default — async job
```bash cURL theme={null}
# 1. Submit
curl -X POST "https://api.yto-01.edge.polargrid.ai/v1/audio/transcriptions?model=whisper-large-v3-turbo&language=en" \
-H "Authorization: Bearer pg_your_api_key" \
-F "file=@recording.mp3"
# {
# "job_id": "job_transcription_a1b2c3d4e5f6",
# "status": "accepted",
# "poll_url": "/v1/audio/transcriptions?job_id=job_transcription_a1b2c3d4e5f6"
# }
# 2. Poll
curl "https://api.yto-01.edge.polargrid.ai/v1/audio/transcriptions?job_id=job_transcription_a1b2c3d4e5f6" \
-H "Authorization: Bearer pg_your_api_key"
```
```javascript JavaScript theme={null}
// SDK helper: submit and wait for the result.
const result = await client.transcribeAndWait({
file,
model: 'whisper-large-v3-turbo',
language: 'en',
});
console.log(result.text);
```
```python Python theme={null}
result = await client.transcribe_and_wait(
file=Path("recording.mp3"),
model="whisper-large-v3-turbo",
language="en",
)
print(result.text)
```
#### Streaming — SSE
```bash cURL theme={null}
curl -N -X POST "https://api.yto-01.edge.polargrid.ai/v1/audio/transcriptions?stream=true&model=whisper-large-v3-turbo&language=en" \
-H "Authorization: Bearer pg_your_api_key" \
-F "file=@recording.mp3"
```
```javascript JavaScript theme={null}
for await (const evt of client.transcribeStream({
file,
model: 'whisper-large-v3-turbo',
language: 'en',
})) {
if (evt.type === 'transcript.text.delta') console.log('[partial]', evt.delta);
if (evt.type === 'transcript.text.done') console.log(evt.text); // authoritative
}
```
```python Python theme={null}
async for evt in client.transcribe_stream(
Path("recording.mp3"),
TranscriptionRequest(model="whisper-large-v3-turbo", language="en"),
):
if evt.type == "transcript.text.delta":
print(f"[partial] {evt.delta}") # provisional; see Delta semantics
elif evt.type == "transcript.text.done":
print(evt.text) # authoritative transcript
```
```bash CLI theme={null}
polargrid transcribe recording.mp3 --model whisper-large-v3-turbo --stream
```
#### Sync — blocking request
```bash cURL theme={null}
curl -X POST "https://api.yto-01.edge.polargrid.ai/v1/audio/transcriptions?sync=true&model=whisper-large-v3-turbo&language=en" \
-H "Authorization: Bearer pg_your_api_key" \
-F "file=@recording.mp3"
```
```javascript JavaScript theme={null}
const transcription = await client.transcribe({
file,
model: 'whisper-large-v3-turbo',
language: 'en',
stream: false, // hits the ?sync=true path
});
console.log(transcription.text);
```
```python Python theme={null}
transcription = await client.transcribe(
file=Path("recording.mp3"),
request=TranscriptionRequest(
model="whisper-large-v3-turbo",
language="en",
stream=False,
),
)
print(transcription.text)
```
#### Cohere model — same surface, different model id
`cohere-transcribe-03-2026` supports the same
sync, stream, and async modes. Swap the `model` query parameter -- everything
else is identical:
```bash cURL theme={null}
curl -N -X POST "https://api.yto-01.edge.polargrid.ai/v1/audio/transcriptions?stream=true&model=cohere-transcribe-03-2026" \
-H "Authorization: Bearer pg_your_api_key" \
-F "file=@recording.mp3"
```
```javascript JavaScript theme={null}
for await (const evt of client.transcribeStream({
file,
model: 'cohere-transcribe-03-2026',
})) {
if (evt.type === 'transcript.text.delta') console.log('[partial]', evt.delta);
if (evt.type === 'transcript.text.done') console.log(evt.text);
}
```
```python Python theme={null}
async for evt in client.transcribe_stream(
Path("recording.mp3"),
TranscriptionRequest(model="cohere-transcribe-03-2026"),
):
if evt.type == "transcript.text.delta":
print(f"[partial] {evt.delta}")
elif evt.type == "transcript.text.done":
print(evt.text)
```
Cohere covers 14 languages but does **not** auto-detect: it requires a language to decode and falls back to `en` when `language` is omitted. The audio is still transcribed correctly (the model is multilingual), but the `language` field in the response will report `en` rather than the spoken language. Pass `language` explicitly (e.g. `&language=fr`) whenever you need the response metadata to reflect the actual language.
### Live Streaming — WebSocket
```
wss://api..edge.polargrid.ai/v1/audio/transcriptions/ws
```
The upload modes above need the complete file before transcription starts.
The WebSocket surface transcribes **while you capture**: stream PCM frames
from the microphone and partial transcripts arrive during the utterance.
* Connect with `?token=`; optional `model`,
`language`, `prompt`, `window_s` query params
* Send **binary** frames: 16 kHz mono little-endian 16-bit PCM
* Send `{"type":"stop"}` (text frame) to end the utterance
* Receive the same event vocabulary as SSE: `transcript.text.delta` per
\~1 s window of new audio, then one authoritative `transcript.text.done`
after `stop` (see [Delta semantics](#delta-semantics) — deltas are
provisional display hints, not concatenable fragments)
```python theme={null}
import asyncio, json, websockets
async def live_transcribe(frames):
url = "wss://api.yto-01.edge.polargrid.ai/v1/audio/transcriptions/ws" \
"?token=pg_your_api_key&model=whisper-large-v3-turbo"
async with websockets.connect(url) as ws:
async def send():
async for frame in frames: # 16kHz mono int16 PCM chunks
await ws.send(frame)
await ws.send(json.dumps({"type": "stop"}))
async def recv():
async for msg in ws:
ev = json.loads(msg)
if ev["type"] == "transcript.text.delta":
print(f"[partial] {ev['delta']}") # provisional; see Delta semantics
elif ev["type"] == "transcript.text.done":
return ev["text"] # authoritative transcript
_, text = await asyncio.gather(send(), recv())
return text
```
Sessions are capped at 120 seconds of audio — this surface targets
conversational turns. For long recordings use the upload endpoint.
Measured on a production edge (100 runs, 5-clip 4.2–7.7 s corpus streamed at
real-time mic pace): first partial arrives \~1 s into the utterance (the first
window boundary), subsequent partials each window while audio is still
flowing, and the authoritative `done` lands **159 ms p50 / 205 ms p95 after
the `stop` marker** — the effective end-of-speech-to-transcript latency for a
voice agent on this surface. See the
[whisper-large-v3-turbo model card](/models/whisper-large-v3-turbo) for the
full table.
### SSE Event Types
| `type` | Fields | When |
| ----------------------- | ---------------------------------- | ----------------------------------------------------- |
| `transcript.text.delta` | `delta` (string) | Zero or more, provisional (see Delta semantics below) |
| `transcript.text.done` | `text`, `duration` (s), `language` | Exactly once, before the stream closes |
| `error` | `error` (string) | On failure — stream ends after |
Terminated by `data: [DONE]\n\n` (SSE surface only).
### Delta semantics
Applies to both the SSE and WebSocket surfaces, which share the same engine.
Each window re-transcribes the full audio buffered so far, producing a new
hypothesis. The `delta` field then carries **one of two things**:
* the **new suffix**, when the new hypothesis extends the previous one
unchanged (e.g. `" This is a short."`), or
* the **full revised hypothesis**, when re-decoding changed earlier text
(e.g. `"Hello. This is a short test phrase for"`).
The two cases are not distinguishable from the event alone, so **do not
concatenate deltas** (you will render duplicated text) and **do not replace
your display with a bare delta** (you will drop earlier text when the delta
is a suffix). Treat deltas as low-latency provisional output for live
display, and always take the final transcript from `done.text` — it is the
only authoritative value, produced by a fresh pass over the complete audio.
### Polling Responses
`GET /v1/audio/transcriptions?job_id=...` returns:
* `202` while the job is `accepted` / `processing` — `{ job_id, status, poll_interval_ms }`
* `200` when `completed` — formatted body, with `job_id` and `status: "completed"`
* `400` on `failed` / `cancelled`
## Supported Audio Formats
MP3, WAV, M4A, OGG, FLAC, WebM
## Limits
| Limit | Value | On exceed |
| ------------------- | ---------- | ------------------------------------------------------------------- |
| Maximum upload size | **100 MB** | `413` with a structured JSON error (`error.code: "file_too_large"`) |
```json theme={null}
{
"message": "File too large. Maximum upload size is 100MB. For longer recordings, split the audio into segments and submit them as separate requests.",
"error": {
"message": "File too large. Maximum upload size is 100MB. ...",
"type": "invalid_request_error",
"code": "file_too_large"
}
}
```
There is no separate duration limit — only size. As a rule of thumb, 100 MB
holds roughly 1.5 hours of 16 kHz mono WAV or many hours of MP3.
For recordings over the limit, split the audio into segments (for example
with `ffmpeg -f segment`), submit each segment as its own request — the
default async mode is built for this — and concatenate the transcripts.
Splitting on silence rather than fixed intervals avoids cutting words in
half.
# Text-to-Speech
Source: https://polargrid.mintlify.app/api-reference/text-to-speech
Convert text to natural-sounding speech
# Text-to-Speech
Generate audio from text. PolarGrid serves `kokoro-82m` (preset voice catalog) and `tada-3b-ml` (voice-cloning). The endpoint returns audio in the container format requested via `response_format` — supported values are `pcm`, `wav`, and `mp3`.
Edge endpoints accept your `pg_*` API key as a bearer token. See [Authentication](/authentication) for details. The cURL examples below pin Toronto (`yto-01`) for concreteness — substitute another region or discover the fastest one via `GET https://autorouter.polargrid.ai/v1/route`. See [API Overview](/api-reference/overview#picking-a-region) for both patterns.
## Create Speech
```
POST /v1/audio/speech
```
Generate audio from input text.
### Request Body
| Parameter | Type | Required | Default | Description |
| ------------------ | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model` | string | Yes | — | TTS model: `kokoro-82m` or `tada-3b-ml` |
| `input` | string | Yes | — | Text to convert |
| `voice` | string | Yes | — | Voice to use (see below) |
| `voice_transcript` | string | No | — | **`tada-3b-ml` only.** The exact text spoken in the `voice` reference clip. Required when `voice` is a URL or base64 WAV; not needed for `voice: "default"`. Ignored by `kokoro-82m`. |
| `language` | string | No | `en` | **`tada-3b-ml` only.** Target synthesis language: `en`, `fr`, `de`, `es`, `it`, `pt`, `pl`, `ja`, `ar`, `zh`. Ignored by `kokoro-82m`. |
| `response_format` | string | No | `pcm` | Container format: `pcm`, `wav`, or `mp3`. See [Audio Format](#audio-format) below. |
| `stream` | boolean | No | `false` | When `true`, returns chunked audio. See [Streaming](#streaming). Only `pcm` and `opus` are valid `response_format` values when streaming. |
| `speed` | number | No | 1.0 | Speed multiplier (0.25-4.0). `tada-3b-ml` honors this only in batch mode — streaming TADA requires `speed: 1.0`. |
The gateway returns `400 Bad Request` if `input` is empty/whitespace-only, `voice` is empty, or `response_format` is anything other than `pcm`, `wav`, or `mp3`. Synthesis failures (invalid voice ID, upstream error) return `502 Bad Gateway` — never an empty `200`.
#### Input length limits
`input` length is capped per model and enforced **before** synthesis; over-limit requests return `413 Payload Too Large` with a message naming the model and its limit (e.g. `Input too long: maximum 850 characters for tada-3b-ml`). The limit is applied after the gateway strips surrounding quotes and code/markdown artifacts, so it counts the text actually synthesized.
| Model | Max `input` characters |
| ------------ | ---------------------- |
| `tada-3b-ml` | 850 |
| `kokoro-82m` | 4096 |
`tada-3b-ml` has a lower limit because longer inputs can exhaust GPU memory mid-synthesis. The 850-character cap is a deterministic, documented contract that keeps identical requests behaving identically regardless of server load. Split longer text into multiple requests and concatenate the audio client-side.
### Voices
`tada-3b-ml` is a voice-cloning model — see the [model page](/models#hume-ai-tada) for how to provide a reference voice; it does not use the preset voice IDs below.
The `kokoro-82m` model exposes eight preset voices through the PolarGrid SDKs:
| Voice ID | Accent / gender |
| ------------- | ------------------------ |
| `af_bella` | American English, female |
| `af_sarah` | American English, female |
| `am_adam` | American English, male |
| `am_michael` | American English, male |
| `bf_emma` | British English, female |
| `bf_isabella` | British English, female |
| `bm_george` | British English, male |
| `bm_lewis` | British English, male |
See the [Voice AI guide](/guides/voice) for how these map to Kokoro-82M and a link to the full upstream voice list.
**TADA output is deterministic.** `tada-3b-ml` uses diffusion-based synthesis with a fixed random seed, so the same `input` text + same `voice` reference always produces byte-identical audio. This is by design -- the fixed seed guarantees consistent voice identity and timing across requests, which matters for voice-agent pipelines where prosody shifts between calls would be jarring. There is no server-side cache: every request runs full inference and is billed accordingly, even when the output matches a previous call. A user-controllable `seed` parameter is planned for a future API version to let callers introduce deliberate prosody variation.
### Audio Format
Audio is generated at **24 kHz, 16-bit, mono**. The container is chosen by `response_format`:
| `response_format` | Content-Type | Body | Streaming? |
| ----------------- | ------------ | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `pcm` *(default)* | `audio/pcm` | Raw signed 16-bit little-endian samples, no header. | Yes — chunks stream as they're synthesized. |
| `wav` | `audio/wav` | A standard RIFF/WAVE container wrapping the PCM samples. | No — buffered until synthesis completes (a valid WAV header needs the total sample count). |
| `mp3` | `audio/mpeg` | MP3 at 128 kbps CBR (encoded server-side via `libmp3lame`). | No — buffered, then encoded in one shot. |
PCM is the lowest-latency choice and the recommended format for real-time voice-agent pipelines. Pick `wav` if you need a playable file with no client-side post-processing, or `mp3` if bandwidth matters more than first-byte latency.
**REST and SDK defaults differ.** Raw HTTP requests to `/v1/audio/speech` default to `audio/pcm` (headerless 24 kHz 16-bit LE samples) when `response_format` is omitted. The PolarGrid JavaScript and Python SDKs default batch requests to `mp3` and streaming requests to `opus` for OpenAI-style behavior. If you switch between raw `curl`/HTTP and an SDK without setting `response_format` explicitly, you will get different audio containers. Always pass `response_format` to get deterministic output regardless of calling method.
`opus`, `aac`, and `flac` from the OpenAI spec are **not yet supported** in batch mode — requesting them returns `400`. For streaming, `opus` is supported (see below); for batch, transcode PCM client-side if you need one of those:
```bash theme={null}
ffmpeg -f s16le -ar 24000 -ac 1 -i speech.pcm speech.opus
```
### Streaming
Pass `stream: true` to receive chunked audio over a single HTTP response — first bytes typically arrive in under 300 ms, well before synthesis finishes. The PolarGrid SDKs default streaming requests to `response_format: 'opus'`; raw HTTP callers get `pcm` when `response_format` is omitted (the gateway's lowest-latency default).
**Streamable formats:**
| `response_format` | Content-Type | Streamable? | Use when |
| ----------------- | ------------------------ | ----------- | -------------------------------------------------------------- |
| `pcm` | `audio/pcm` | ✓ chunked | Real-time voice-agent pipelines — lowest first-byte latency. |
| `opus` | `audio/ogg; codecs=opus` | ✓ chunked | Bandwidth-constrained clients — server-side encoded at 48 kHz. |
| `wav` | — | ✗ | RIFF header needs the full sample count up front. |
| `mp3` | — | ✗ | Frame alignment incompatible with sub-300 ms TTFB. |
Streaming `wav` or `mp3` returns `400 Bad Request`.
**Streamable models:**
| Model | Streaming? | Notes |
| ------------ | --------------- | --------------------------------------------------------------------- |
| `kokoro-82m` | ✓ `pcm`, `opus` | — |
| `tada-3b-ml` | ✓ `pcm`, `opus` | Per-token via decoupled Triton handler. See TADA speed warning below. |
**TADA streaming: `speed` must be 1.0.** The `tada-3b-ml` model does not support `speed` values other than `1.0` in streaming mode. Setting any other value (e.g., `speed: 1.5`) returns a `400 Bad Request` error from the gateway. If you need speed control with TADA, use batch mode (`stream: false`) instead.
**Response headers:**
* `X-Polargrid-Stream: 1` — set on every streaming response.
* `X-Polargrid-Sample-Rate: 24000` — PCM sample rate; Opus is resampled to 48 kHz inside the Ogg container.
#### Detecting truncated streams
A mid-stream Triton or upstream failure closes the connection cleanly — there is no in-band error frame. Clients observe one of:
* A `ReadError` / `IncompleteRead` / `ChunkedEncodingError` raised by the HTTP client.
* For `opus`, an Ogg stream that never sees the `end-of-stream` flag on its final page.
Treat any of these as a synthesis failure and retry. The PolarGrid SDKs surface these as exceptions from the async iterator; they do not silently terminate.
#### Streaming example
```bash cURL theme={null}
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
--no-buffer \
-d '{
"model": "kokoro-82m",
"input": "Streaming hello from PolarGrid!",
"voice": "af_bella",
"response_format": "opus",
"stream": true
}' \
--output stream.ogg
```
```javascript JavaScript theme={null}
for await (const chunk of client.textToSpeechStream({
model: 'kokoro-82m',
input: 'Streaming hello from PolarGrid!',
voice: 'af_bella',
responseFormat: 'opus',
})) {
audioPlayer.appendChunk(chunk);
}
```
```python Python theme={null}
async for chunk in client.text_to_speech_stream({
"model": "kokoro-82m",
"input": "Streaming hello from PolarGrid!",
"voice": "af_bella",
"response_format": "opus",
}):
audio_player.append_chunk(chunk)
```
**Not supported in v1:**
* **Cartesia-compatible WebSocket TTS** (`wss://api.cartesia.ai/tts/websocket` shape). PolarGrid streaming TTS uses chunked HTTP only. Customers porting from Cartesia must swap the transport layer.
* **WebSocket TTS endpoint of any kind.** There is no `/v1/audio/speech/ws`. PolarGrid exposes WebSockets only for completions (`/v1/completions/ws`) and PersonaPlex's full-duplex voice pipeline.
* **Streaming WAV and MP3.** WAV requires the full sample count for its RIFF header; MP3 frame alignment can't hit sub-300 ms TTFB. Request `pcm` or `opus` for streaming.
If your existing pipeline depends on any of the above, file a request in the [PolarGrid roadmap](https://github.com/PolarGrid-AI/polargrid-monorepo/issues).
### Example Request
```bash cURL theme={null}
# Edge endpoints accept your pg_* API key as a bearer token — see Authentication
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer pg_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"model": "kokoro-82m",
"input": "Hello from PolarGrid!",
"voice": "af_bella",
"response_format": "wav"
}' \
--output speech.wav
```
```javascript JavaScript theme={null}
const audioBuffer = await client.textToSpeech({
model: 'kokoro-82m',
input: 'Hello from PolarGrid!',
voice: 'af_bella',
responseFormat: 'wav',
});
// Fully-formed RIFF/WAVE container — playable directly in browsers and
// most audio libraries with no post-processing.
import { writeFile } from 'fs/promises';
await writeFile('speech.wav', Buffer.from(audioBuffer));
```
```python Python theme={null}
audio_buffer = await client.text_to_speech({
"model": "kokoro-82m",
"input": "Hello from PolarGrid!",
"voice": "af_bella",
"response_format": "wav",
})
with open("speech.wav", "wb") as f:
f.write(audio_buffer)
```
### Response
Returns the requested container as a binary body, with `Content-Type` set to `audio/pcm`, `audio/wav`, or `audio/mpeg` to match `response_format`.
See [Streaming](#streaming) above for the streaming TTS contract, supported formats and models, and code samples.
# Authentication
Source: https://polargrid.mintlify.app/authentication
How to authenticate with the PolarGrid API
# Authentication
All PolarGrid API requests require authentication using an API key.
## API Keys
API keys are created in the PolarGrid Console and start with `pg_`.
### Creating a Key
1. Open the API Keys page — either click **Generate API Key** (or **Manage API Keys**) on the Dashboard Overview, or go to **Settings → API Keys** in the sidebar
2. Click **Generate New Key**
3. Give it a descriptive name (e.g., "Production App", "Development")
4. Select permissions: `read-write` (default), `read-only`, or `admin`
5. Copy the key immediately — it won't be shown again
### Using Your Key
The SDKs handle authentication automatically — just pass your API key:
```javascript JavaScript theme={null}
const client = await PolarGrid.create({
apiKey: 'pg_your_api_key',
});
```
```python Python theme={null}
client = await PolarGrid.create(api_key="pg_your_api_key")
```
### Raw HTTP (cURL)
Send your `pg_*` API key directly to the edge. Pick a region with the autorouter (one request) or pin one.
```bash theme={null}
export API_KEY="pg_your_api_key"
# Option A — auto: ask the autorouter for the best edge for the caller.
# Returns {region, name, endpoint, ttl}. The endpoint is the actual base URL.
EDGE=$(curl -s https://autorouter.polargrid.ai/v1/route | jq -r .endpoint)
# Option B — pinned: hardcode a region instead
# EDGE="https://api.yto-01.edge.polargrid.ai"
# Call the edge directly with your API key.
curl -s -X POST "$EDGE/v1/chat/completions" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "qwen-3.8-27b", "messages": [{"role": "user", "content": "Hi"}]}'
```
The autorouter (`autorouter.polargrid.ai`) is a discovery endpoint — it serves `GET /v1/route` only. It does not proxy `/v1/chat/completions`, `/v1/models`, or any other inference traffic; POST against it is rejected by CloudFront with 403.
## Playground vs API
The **Playground** in the Console uses your **real `pg_*` API key** — the same key from [Creating a Key](#creating-a-key) above. There is no separate "playground token".
Click **Playground** in the Console sidebar.
Pick a saved key from the Playground's API key dropdown, or paste one manually.
Under the hood, the Playground sends your API key directly to the nearest edge — the same flow as a raw cURL call.
## Environment Variables
We recommend storing your API key in an environment variable:
```bash theme={null}
export POLARGRID_API_KEY="pg_your_api_key"
```
The SDKs automatically read from this variable:
```javascript JavaScript theme={null}
// No need to pass apiKey if POLARGRID_API_KEY is set
const client = await PolarGrid.create();
```
```python Python theme={null}
# No need to pass api_key if POLARGRID_API_KEY is set
client = await PolarGrid.create()
```
## Permission Levels
| Level | Description |
| ------------ | ----------------------------------------------------------------------------------------------------------- |
| `read-write` | Default for keys minted from the Console. Recommended for typical SDK / API usage. |
| `read-only` | Same data-plane access today; intended for callers that should not mint LiveKit room tokens (`/v1/tokens`). |
| `admin` | Same data-plane access plus the ability to mint LiveKit room tokens. |
The inference endpoints (`/v1/chat/completions`, `/v1/completions`, `/v1/models`, `/v1/audio/*`) are not currently scope-gated — any active key can call them. Scope enforcement is presently scoped to `/v1/tokens`. Default to `read-write` unless you have a specific reason to narrow.
## Troubleshooting Auth Errors
| HTTP Code | Error | Cause | Fix |
| --------- | -------------------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `401` | `Invalid API key` | API key not recognized, revoked, or deleted | Verify the key is correct and active in the Console. If revoked, generate a new key |
| `403` | `{"Message": null}` or Forbidden | Request blocked by infrastructure (CloudFront/WAF) before reaching the auth service | Ensure you're using the correct endpoint URL and that your request includes valid headers. See [Regions](/guides/regions) |
## Advanced: Session Tokens (CLI / Management Plane)
For most users, the `pg_*` API key sent as `Authorization: Bearer pg_*` is all you need -- edge endpoints accept it directly and no token exchange is required.
The CLI and management-plane APIs (org management, key listing) use a separate session-token flow: the CLI exchanges a `pg_*` key via `POST https://auth.polargrid.ai/v1/auth/session` to obtain a short-lived JWT for those endpoints. This is handled automatically by `polargrid login` and is not needed for inference.
| Token | Default Lifetime | Purpose |
| ------------- | ---------------- | --------------------------------------------- |
| Session (JWT) | 24 hours | Authorizes CLI / management-plane requests |
| Refresh | 30 days | Extends the session without re-authenticating |
Edge inference endpoints (`/v1/chat/completions`, `/v1/audio/*`, `/v1/models`) do **not** require a JWT -- send your `pg_*` API key directly as a bearer token. The session-token flow above applies only to CLI and management-plane operations.
## Security Best Practices
Never commit API keys to source control or expose them in client-side code.
* Use environment variables for API keys
* Rotate keys periodically
* Use separate keys for development and production
* Revoke keys immediately if compromised
## CLI Authentication
The CLI supports two authentication methods:
### Browser Login (Interactive)
```bash theme={null}
polargrid login
```
Opens your browser for OAuth authentication. Best for development.
### API Key (CI/CD)
```bash theme={null}
export POLARGRID_API_KEY="pg_your_api_key"
polargrid test inference --region toronto --prompt "Hello"
```
No login needed — just set the environment variable.
# Managing Context in Long Conversations
Source: https://polargrid.mintlify.app/guides/context-management
Stay under the model context window in multi-turn and voice agent workloads
# Managing Context in Long Conversations
Every LLM on PolarGrid enforces a hard context limit covering the system prompt, the full message history, and the requested completion. For `qwen-3.8-27b` it is **262,144 tokens** (256K). There is no server-side truncation: when a request exceeds the limit, the API returns:
```json theme={null}
{
"error": {
"message": "Request exceeds the model's context window of 262144 tokens. Reduce the prompt or conversation history (for example, truncate or summarize older turns) and retry.",
"type": "invalid_request_error"
}
}
```
A retry without shrinking the request can never succeed — treat this 400 as a signal to compact history, not as a transient failure.
## Why this matters for voice agents
Voice sessions accumulate context fast: every user turn and assistant reply joins the history. A 256K window is unlikely to be exhausted by a single call, so the pressure here is **latency and cost**, not the hard cap — time-to-first-token grows with prompt length, and you pay for every input token on every turn. Compact history because it keeps the agent responsive and cheap, not because the session is about to die.
If you pin a lower `max_model_len` on a node, or move to a model with a smaller window, the hard cap becomes the binding constraint again — the patterns below cover both cases.
## Rule of thumb for budgeting
English text averages roughly 4 characters (≈0.75 words) per token. `qwen-3.8-27b`'s 256K window is far larger than a voice conversation needs, so budget for **responsiveness** rather than to the ceiling:
| Slice | Suggested budget |
| ------------------------------------- | ----------------------------------------------------------- |
| System prompt | ≤ 1,000 tokens |
| Reserved for the reply (`max_tokens`) | 512–1,024 tokens |
| Conversation history | \~6,000 tokens (≈ 45 average voice turns) before compacting |
These are latency targets, not limits. Track an estimate as you go (`len(text) / 4` is adequate) and compact when history crosses your budget. If you are running against a genuinely small window, compact at \~80% of the hard limit rather than reacting to the 400.
## Pattern 1: Sliding window (simplest, fits most voice agents)
Keep the system prompt plus the most recent N turns; drop the oldest user/assistant pairs first.
```python theme={null}
MAX_HISTORY_TOKENS = 6000
def estimate_tokens(messages):
return sum(len(m["content"]) // 4 for m in messages)
def windowed(system_prompt, history):
msgs = list(history)
while msgs and estimate_tokens(msgs) > MAX_HISTORY_TOKENS:
# Drop the oldest exchange; never drop the system prompt.
msgs = msgs[2:] if len(msgs) >= 2 else []
return [{"role": "system", "content": system_prompt}, *msgs]
```
Good enough whenever the conversation's relevant state lives in recent turns — appointment booking, support triage, order taking.
## Pattern 2: Running summary (when early context must survive)
When facts from early in the call matter at the end (caller name, account details, the original problem statement), fold older turns into a compact summary instead of dropping them:
1. When history crosses your threshold, send the oldest turns to the LLM with a "summarize the key facts in under 100 words" instruction.
2. Replace those turns with a single system-adjacent message: `{"role": "system", "content": "Summary of earlier conversation: ..."}`.
3. Keep the most recent turns verbatim.
This costs one extra LLM call per compaction but caps history growth permanently. For voice, run the compaction during the user's speaking turn so it never adds response latency.
## Pattern 3: Structured state instead of raw history
Voice agents that collect fields (name, phone, address, intent) often don't need the transcript at all — extract entities into a structured object as you go and prompt with the state object plus only the last 2-3 turns. Smallest possible context, immune to call length.
## Handling the overflow error defensively
Even with budgeting, guard the call:
```python theme={null}
try:
response = client.chat.completions.create(model="qwen-3.8-27b", messages=msgs, max_tokens=512)
except APIError as e:
if "context window" in str(e):
msgs = compact(msgs) # your sliding-window or summary step
response = client.chat.completions.create(model="qwen-3.8-27b", messages=msgs, max_tokens=512)
else:
raise
```
Do **not** put context-overflow 400s through generic retry/backoff — they are deterministic.
## Per-model limits
Check the model card for each model's context window ([Models](/models)); the limit also applies to `/v1/completions` prompts. For workloads that genuinely need very long context, [contact us](mailto:hello@polargrid.ai) about enterprise configurations.
# Error Handling
Source: https://polargrid.mintlify.app/guides/error-handling
Handle errors gracefully in your application
# Error Handling
The PolarGrid SDKs provide typed errors for robust error handling.
## Error Types
| Error | Status | Description |
| --------------------- | ------ | ------------------------------------------ |
| `AuthenticationError` | 401 | Invalid or expired API key |
| `ValidationError` | 400 | Invalid request parameters |
| `NotFoundError` | 404 | Resource not found |
| `RateLimitError` | 429 | [Rate limit](/guides/rate-limits) exceeded |
| `ServerError` | 5xx | Server-side error |
| `NetworkError` | — | Connection failed |
| `TimeoutError` | — | Request timed out |
## Basic Error Handling
```javascript JavaScript theme={null}
import {
PolarGrid,
isPolarGridError,
AuthenticationError,
ValidationError,
RateLimitError,
NetworkError,
TimeoutError,
} from '@polargrid/polargrid-sdk';
try {
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Hello' }],
});
} catch (error) {
if (isPolarGridError(error)) {
console.error(`Error: ${error.message}`);
console.error(`Request ID: ${error.requestId}`);
if (error instanceof AuthenticationError) {
// Redirect to login or refresh API key
console.error('Please check your API key');
} else if (error instanceof ValidationError) {
// Fix request parameters
console.error('Invalid parameters:', error.details);
} else if (error instanceof RateLimitError) {
// Wait and retry
console.error(`Retry after ${error.retryAfter} seconds`);
} else if (error instanceof NetworkError) {
// Check connection
console.error('Network error - check your connection');
} else if (error instanceof TimeoutError) {
// Increase timeout or retry
console.error('Request timed out');
}
} else {
// Unknown error
throw error;
}
}
```
```python Python theme={null}
from polargrid import (
PolarGrid,
PolarGridError,
AuthenticationError,
ValidationError,
RateLimitError,
NetworkError,
TimeoutError,
)
try:
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello"}],
})
except PolarGridError as e:
print(f"Error: {e.message}")
print(f"Request ID: {e.request_id}")
if isinstance(e, AuthenticationError):
print("Please check your API key")
elif isinstance(e, ValidationError):
print(f"Invalid parameters: {e.details}")
elif isinstance(e, RateLimitError):
print(f"Retry after {e.retry_after} seconds")
elif isinstance(e, NetworkError):
print("Network error - check your connection")
elif isinstance(e, TimeoutError):
print("Request timed out")
```
## Retry Logic
The SDKs include automatic retry with exponential backoff for transient errors. Configure with:
```javascript theme={null}
const client = new PolarGrid({
apiKey: 'pg_...',
maxRetries: 3, // Default: 3
timeout: 30000, // Default: 30s
});
```
### Custom Retry
For more control:
```javascript theme={null}
async function withRetry(fn, maxRetries = 3) {
let lastError;
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
return await fn();
} catch (error) {
lastError = error;
// Don't retry auth errors
if (error instanceof AuthenticationError) {
throw error;
}
// Don't retry validation errors
if (error instanceof ValidationError) {
throw error;
}
// Exponential backoff
const delay = Math.pow(2, attempt) * 1000;
await new Promise(r => setTimeout(r, delay));
}
}
throw lastError;
}
// Usage
const response = await withRetry(() =>
client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Hello' }],
})
);
```
## Rate Limiting
When rate limited, the `RateLimitError` includes a `retryAfter` property (populated when the server sends a `Retry-After` header; otherwise `undefined`, so fall back to a fixed delay as shown below). PolarGrid allows 100 requests per minute per user, counted independently on each edge node. See the [Rate Limits guide](/guides/rate-limits) for full details, retry strategies, and production best practices.
```javascript theme={null}
try {
const response = await client.chatCompletion(request);
} catch (error) {
if (error instanceof RateLimitError) {
const waitMs = (error.retryAfter || 60) * 1000;
await new Promise(r => setTimeout(r, waitMs));
// Retry request
}
}
```
## Request IDs
Every request includes a unique ID for debugging. Include it when contacting support:
```javascript theme={null}
try {
const response = await client.chatCompletion(request);
} catch (error) {
if (isPolarGridError(error)) {
console.error(`Request ID for support: ${error.requestId}`);
}
}
```
## Validation Errors
Validation errors include details about what's wrong:
```javascript theme={null}
try {
await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [], // Empty messages array
});
} catch (error) {
if (error instanceof ValidationError) {
// error.message: "Messages array is required and cannot be empty"
// error.details: { field: 'messages', reason: 'empty' }
}
}
```
## Debug Mode
Enable debug logging to see request/response details:
```javascript theme={null}
const client = new PolarGrid({
apiKey: 'pg_...',
debug: true,
});
// Logs:
// [PolarGrid] Making request (attempt 1): url=..., method=POST
// [PolarGrid] Request successful: requestId=..., status=200
```
# Migrating from OpenAI
Source: https://polargrid.mintlify.app/guides/migration
Switch from OpenAI to PolarGrid with minimal code changes
# Migrating from OpenAI
PolarGrid exposes an OpenAI-compatible API. If you're already using OpenAI, migration is straightforward — in many cases it's a single line change.
## Option 1: Use the OpenAI SDK directly
The fastest migration path. Keep using the OpenAI SDK — just change the base URL and auth.
```javascript JavaScript theme={null}
import OpenAI from 'openai';
// Before: OpenAI
const openai = new OpenAI({ apiKey: 'sk-...' });
// After: PolarGrid (using OpenAI SDK)
const pg = new OpenAI({
apiKey: 'pg_your_api_key', // PolarGrid API key — sent directly to the edge
baseURL: 'https://api.yto-01.edge.polargrid.ai/v1', // pin a region, or discover one via the autorouter (see /guides/regions)
});
// Same API — no other changes needed
const response = await pg.chat.completions.create({
model: 'qwen-3.8-27b', // PolarGrid model name
messages: [{ role: 'user', content: 'Hello!' }],
});
```
```python Python theme={null}
from openai import OpenAI
# Before: OpenAI
client = OpenAI(api_key="sk-...")
# After: PolarGrid (using OpenAI SDK)
client = OpenAI(
api_key="pg_your_api_key", # PolarGrid API key — sent directly to the edge
base_url="https://api.yto-01.edge.polargrid.ai/v1", # pin a region, or discover one via the autorouter (see /guides/regions)
)
# Same API — no other changes needed
response = client.chat.completions.create(
model="qwen-3.8-27b", # PolarGrid model name
messages=[{"role": "user", "content": "Hello!"}],
)
```
The edge accepts your `pg_*` API key directly — no token exchange step.
## Option 2: Use the PolarGrid SDK
The PolarGrid SDK handles authentication, region selection, and token refresh automatically.
```javascript JavaScript theme={null}
import { PolarGrid } from '@polargrid/polargrid-sdk';
// Auto-selects fastest region; sends your API key directly to the edge.
const client = await PolarGrid.create({
apiKey: 'pg_your_api_key',
});
// Same familiar API shape
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(response.choices[0].message.content);
```
```python Python theme={null}
from polargrid import PolarGrid
# Auto-selects fastest region; sends your API key directly to the edge.
client = await PolarGrid.create(api_key="pg_your_api_key")
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}],
})
print(response.choices[0].message.content)
```
**What the SDK handles for you:**
* Latency-based region selection (pings all regions, picks the fastest)
* Direct API key auth — your `pg_*` is the bearer token, no extra exchange step
* Streaming, audio, and model management APIs
## Option 3: Direct HTTP
If you're calling OpenAI via raw HTTP, swap the base URL:
```bash theme={null}
# Before: OpenAI
curl https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer sk-..." \
-H "Content-Type: application/json" \
-d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'
# After: PolarGrid — API key direct to the edge
# Step 1: Ask the autorouter which edge is best for the caller (returns
# {region, name, endpoint, ttl}; endpoint is the actual base URL).
EDGE=$(curl -s https://autorouter.polargrid.ai/v1/route | jq -r .endpoint)
# Step 2: POST inference directly to that edge with your pg_* key
curl $EDGE/v1/chat/completions \
-H "Authorization: Bearer pg_your_api_key" \
-H "Content-Type: application/json" \
-d '{"model": "qwen-3.8-27b", "messages": [{"role": "user", "content": "Hello"}]}'
```
## What's Different
| | OpenAI | PolarGrid |
| ------------------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Auth** | Single API key | Single API key (`pg_*`) sent directly to the edge |
| **Base URL** | `api.openai.com` | `api.{region}.edge.polargrid.ai` (discover the best region via `GET https://autorouter.polargrid.ai/v1/route`, or pin one) |
| **Models** | GPT-4o, GPT-4o-mini, etc. | Qwen, Llama, Whisper, etc. ([full list](/models)) |
| **Regions** | Single endpoint | Multiple edge regions ([see regions](/guides/regions)) |
| **Streaming** | SSE | SSE (same format) |
| **Vision input** | `image_url` content parts | Same `image_url` content-parts shape — but only on vision-capable models (`qwen-3.6-35b-a3b` on the `sfo-02` staging node). Text-only models return `400` for image content. See [Chat Completions → Vision](/api-reference/chat-completions#vision-image-input). |
| **Response format** | OpenAI JSON | Same OpenAI-compatible JSON |
| **Audio** | `/v1/audio/speech`, `/v1/audio/transcriptions` | Same endpoints; transcriptions are **async by default** for raw HTTP callers — see below |
The PolarGrid SDK eliminates most of these differences — it handles auth, region selection, and token refresh automatically. If you're building a new integration, start with the SDK.
## Speech-to-Text: Sync vs Async
OpenAI's transcription endpoint is strictly synchronous. PolarGrid's `/v1/audio/transcriptions` defaults to an **async job** — `202 Accepted` with `{ job_id, status, poll_url }` — which is better for long files but surprises OpenAI migrators expecting a transcript in the response body.
What you'll get depends on how you call it:
* **Stock OpenAI SDK (Option 1 above): no change needed.** OpenAI SDKs send `model` as a multipart form field, and the endpoint serves those requests synchronously — you get the transcript inline, exactly as you do against OpenAI.
* **Raw HTTP or query-parameter requests: add `?sync=true`** to block until completion and get the formatted result inline. Without it you'll receive a JSON job object, not your transcript.
```bash theme={null}
# Raw HTTP migration — note ?sync=true
curl "$EDGE/v1/audio/transcriptions?model=whisper-large-v3-turbo&sync=true" \
-H "Authorization: Bearer pg_your_api_key" \
-F file=@recording.mp3
```
There's also `?stream=true` for live partial transcripts over SSE (mutually exclusive with `sync=true`). Full details in the [Speech-to-Text API reference](/api-reference/speech-to-text).
## Next Steps
See all available models and specs
Understand edge regions and auto-routing
Stream responses as they're generated
Text-to-speech and speech-to-text
# Model Availability by Region
Source: https://polargrid.mintlify.app/guides/model-availability
Which models are served on which edge regions, and how to verify at runtime
# Model Availability by Region
Models are pre-deployed per edge region — not every model runs everywhere. A request for a model that isn't loaded on the region you call returns `404: Model not loaded`, so check availability before pinning a region.
## Availability Matrix
Verified against live `GET /v1/models` responses on 2026-06-11.
| Model | `yto-01` | `yul-01` | `yvr-02` | `nyc-01` | `nyc-02` | `sfo-01` | `dfw-01` | `dfw-02` |
| --------------------------------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| `qwen-3.8-27b` (LLM) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| `whisper-large-v3-turbo` (STT) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
| `cohere-transcribe-03-2026` (STT) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
| `kokoro-82m` (TTS) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
| `tada-3b-ml` (TTS) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Exceptions to the otherwise-uniform suite — confirm at runtime if you pin them:
* `qwen-3.6-35b-a3b` is in limited availability (customer pilot) and is not currently served from a public region — [contact us](https://polargrid.ai/contact) for access.
* `dfw-02` (Dallas 02) currently serves only `qwen-3.8-27b` and `tada-3b-ml` (no STT, no `kokoro-82m`).
* `lax-01` (Los Angeles) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
* `sea-01` (Seattle) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
* `chi-01` (Chicago) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
* `phx-01` (Phoenix) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
* `was-01` (Washington DC) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
* `mia-01` (Miami) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
* `sfo-03` (San Francisco) came online after this snapshot was taken — verify its loadout at runtime via `GET /v1/models` before pinning it.
If your pipeline depends on a specific model, pin a region that serves it or let the autorouter pick one (see below).
## Verify at Runtime
The matrix above is a snapshot — the authoritative source is always the region itself:
```bash theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/models \
-H "Authorization: Bearer pg_your_api_key"
```
The response lists every model currently loaded on that region. Check this before pinning a region in production, and treat `404: Model not loaded` as a signal to re-check rather than retry.
## Route by Model
The [autorouter](/guides/regions) accepts a `model` filter and only returns regions that serve it:
```bash theme={null}
# Best region that serves qwen-3.8-27b for this caller
curl "https://autorouter.polargrid.ai/v1/route?model=qwen-3.8-27b"
```
Without the filter, the autorouter picks the best region by geography and load — which may not serve the model you need. **If your pipeline requires a specific model (every voice agent does), always pass the `model` parameter.** A `404` from this endpoint means no region currently serves that model.
## Multi-Model Pipelines
A voice pipeline (STT + LLM + TTS) needs every model in the chain on the same region to avoid cross-region hops. Pick the region using the most constrained model — currently the LLM — and the rest of the suite is available everywhere it is.
# No-Code Integrations
Source: https://polargrid.mintlify.app/guides/no-code
Use PolarGrid with n8n, Zapier, Make, and other automation tools
# No-Code Integrations
PolarGrid's OpenAI-compatible API works with any automation tool that supports HTTP requests. This guide shows how to connect from no-code platforms.
## Authentication Setup
PolarGrid edge endpoints accept your `pg_*` API key directly — a single HTTP node is all you need.
### Make Inference Calls
Configure an HTTP Request node:
| Field | Value |
| ---------- | ---------------------------------------------------------- |
| **Method** | POST |
| **URL** | `https://api.yto-01.edge.polargrid.ai/v1/chat/completions` |
| **Header** | `Authorization: Bearer pg_your_api_key` |
| **Header** | `Content-Type: application/json` |
| **Body** | See below |
```json theme={null}
{
"model": "qwen-3.8-27b",
"messages": [
{"role": "user", "content": "Your prompt here"}
]
}
```
## n8n
In n8n, use a single **HTTP Request** node:
* **POST** to `https://api.yto-01.edge.polargrid.ai/v1/chat/completions`
* `Authorization: Bearer pg_your_api_key`
* `Content-Type: application/json`
* Body: the chat completion JSON above
Store the API key in n8n's credentials store rather than hardcoding it in the node.
## Zapier / Make / Other Tools
Any tool with HTTP request support can call PolarGrid edge endpoints in a single step:
1. **HTTP step**: POST to the edge URL with `Authorization: Bearer pg_*`
The key differences per platform:
* **Zapier**: Use a "Webhooks by Zapier" action (Custom Request)
* **Make (Integromat)**: Use an "HTTP — Make a request" module
* **Pipedream**: Use a Node.js or HTTP step
## Available Endpoints
All standard PolarGrid endpoints work from no-code tools:
| Endpoint | Use Case |
| ------------------------------- | --------------------------------------------- |
| `POST /v1/chat/completions` | Chat / text generation |
| `POST /v1/audio/transcriptions` | Speech-to-text (send audio file as form data) |
| `POST /v1/audio/speech` | Text-to-speech (returns audio bytes) |
| `GET /v1/models` | List available models |
## Choosing a Region
Pick the region closest to your automation platform's servers:
| Region | ID | Best For |
| --------- | -------- | ---------------------------- |
| Toronto | `yto-01` | US East, Eastern Canada |
| Vancouver | `yvr-02` | US West, Western Canada |
| Montreal | `yul-01` | Northeast US, Eastern Canada |
Replace the region ID in the URL: `https://api.{region-id}.edge.polargrid.ai`
# PersonaPlex
Source: https://polargrid.mintlify.app/guides/personaplex
Real-time persona-driven voice conversations over WebSocket
# PersonaPlex
PersonaPlex is PolarGrid's real-time voice conversation endpoint. It streams Opus audio in both directions over a single WebSocket, with a persona prompt and voice selected at connect time.
## PersonaPlex vs. the Modular Pipeline Agent
| Dimension | PersonaPlex (multi-modal) | Modular Pipeline Agent |
| ------------------ | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Model architecture | Single multi-modal model (audio in / audio out) | Three models: STT → LLM → TTS |
| Latency profile | One model inference per turn | Three model calls per turn; LLM output is streamed into TTS at sentence boundaries rather than after generation completes |
| Persona / prompt | Text `persona` query parameter at connect time | System prompt passed to the LLM |
| Voice selection | One of the PersonaPlex voice IDs (`NATF0`…) | Any voice supported by the selected TTS model |
| Model substitution | Not supported (models are bundled) | Any STT / LLM / TTS from `GET /v1/models` |
| Barge-in | Handled inside the multi-modal model | User speech cancels in-flight LLM and TTS |
| Structured events | Audio frames + transcript text frames | JSON event stream (see [Modular Pipeline Agent](/guides/voice-agent)) |
| Function calling | Not applicable (audio-native model) | Via the LLM step, once native tool-use is available |
PersonaPlex is **not** listed in `GET /v1/models` — that endpoint only returns the Triton-served request/response models (`qwen-3.8-27b`, `whisper-large-v3-turbo`, `cohere-transcribe-03-2026`, `kokoro-82m`, `tada-3b-ml`). PersonaPlex runs as a separate `moshi-backend` LiveKit agent pod with its own WebSocket endpoint and wire protocol.
## Connecting
Open a WebSocket to the voice endpoint with your `pg_*` API key as the `token` query parameter.
### Open the WebSocket
```
wss://api..edge.polargrid.ai/v1/voice/personaplex
?voice=NATF0
&persona=
&token=pg_your_api_key
```
Default region is `yto-01` (Toronto) if you omit it.
As an alternative to the `token` query parameter, you can pass the credential via the WebSocket subprotocol header:
```
Sec-WebSocket-Protocol: bearer.pg_your_api_key
```
## Voices
Pass one of the following as the `voice` query parameter. **Do not include the `.pt` suffix.**
| Group | Voices |
| ----- | --------------- |
| NATF | `NATF0`–`NATF3` |
| NATM | `NATM0`–`NATM3` |
| VARF | `VARF0`–`VARF4` |
| VARM | `VARM0`–`VARM4` |
## Wire protocol
All frames are **binary**. The first byte is a type tag; the remaining bytes are the payload.
### Client → Server
| Tag | Payload | Notes |
| ------ | ------------------------------ | --------------------------------- |
| `0x01` | Opus audio | Ogg container, mono, 24 kHz |
| `0x02` | UTF-8 text | Inject text into the conversation |
| `0x03` | Control frame (`bos` or `eos`) | Stream boundary markers |
### Server → Client
| Tag | Payload | Notes |
| ------ | ---------- | ------------------------------------ |
| `0x00` | Handshake | Sent once, immediately after connect |
| `0x01` | Opus audio | Generated speech |
| `0x02` | UTF-8 text | Transcript tokens |
**`0x03 eos` marks end-of-turn, not end-of-reply.** Sending `eos` after your audio tells the agent you are done speaking. The agent completes its full reply after receiving `eos`. In builds prior to June 2026, `eos` interrupted the agent's in-flight reply, causing truncated responses. If you integrated during the alpha and added workarounds for that behavior, retest without them.
## Gotchas
**Wait for the `0x00` handshake before sending any audio.** The first audio bytes you send must be the Opus Ogg **BOS (beginning-of-stream) page**. If audio arrives before the handshake — or without a valid BOS page — the server closes the connection with code `1000`.
**Disable your WebSocket library's heartbeat / ping.** The upstream moshi runtime does not respond to RFC 6455 pongs, so a client-side ping timer will tear down an otherwise healthy session. Most libraries expose this as a `ping_interval` or `heartbeat` option — set it to `0` or `None`.
**Sessions are billed by wall-clock duration.** Close the socket as soon as the conversation is idle; an open connection keeps accruing cost even with no audio flowing.
## Quickstart
Both examples below do the same thing: connect to PersonaPlex, send a pre-recorded audio file, and save the response audio to disk. Pick the language you prefer.
### Prerequisites
* A PolarGrid API key ([get one here](https://app.polargrid.ai/dashboard/settings?tab=api-keys))
* An input audio file (WAV, mono, 24 kHz recommended -- other sample rates will be resampled)
### Python
```bash theme={null}
pip install websockets soundfile numpy opuslib
```
```python theme={null}
#!/usr/bin/env python3
"""Send an audio file to PersonaPlex and save the voice response."""
import argparse
import asyncio
import struct
import numpy as np
import opuslib
import soundfile as sf
import websockets
# ---------------------------------------------------------------------------
# Wire protocol tags
# ---------------------------------------------------------------------------
TAG_HANDSHAKE = 0x00 # Server → Client: session ready
TAG_AUDIO = 0x01 # Both directions: Opus audio in Ogg container
TAG_TEXT = 0x02 # Both directions: UTF-8 text
TAG_CONTROL = 0x03 # Client → Server: stream boundary (eos)
# ---------------------------------------------------------------------------
# Ogg helpers — build a minimal Ogg/Opus bitstream from raw Opus frames
# ---------------------------------------------------------------------------
def _ogg_page(serial: int, granule: int, seq: int, bos: bool, eos: bool,
*segments: bytes) -> bytes:
"""Build one Ogg page containing the given segments."""
body = b"".join(segments)
seg_count = len(segments)
seg_table = bytes(len(s) for s in segments)
header_type = (0x02 if bos else 0x00) | (0x04 if eos else 0x00)
# Ogg page header (27 bytes + segment table + body), checksum patched below
header = struct.pack(
"<4sBBqIIIB",
b"OggS", # capture pattern
0, # version
header_type,
granule,
serial,
seq,
0, # checksum placeholder
seg_count,
)
page_no_crc = header + seg_table + body
crc = _ogg_crc(page_no_crc)
return page_no_crc[:22] + struct.pack(" int:
global _OGG_CRC_TABLE
if _OGG_CRC_TABLE is None:
_OGG_CRC_TABLE = []
for i in range(256):
r = i << 24
for _ in range(8):
r = ((r << 1) ^ 0x04C11DB7) & 0xFFFFFFFF if r & 0x80000000 else (r << 1) & 0xFFFFFFFF
_OGG_CRC_TABLE.append(r)
crc = 0
for b in data:
crc = ((crc << 8) ^ _OGG_CRC_TABLE[((crc >> 24) & 0xFF) ^ b]) & 0xFFFFFFFF
return crc
def encode_audio_to_ogg_opus(pcm: np.ndarray, sample_rate: int = 24000,
frame_ms: int = 20) -> bytes:
"""Encode PCM float32 mono audio into an Ogg/Opus bytestream."""
# Resample to 24 kHz if needed
if sample_rate != 24000:
from fractions import Fraction
ratio = Fraction(24000, sample_rate)
n_out = int(len(pcm) * ratio)
indices = np.linspace(0, len(pcm) - 1, n_out)
pcm = np.interp(indices, np.arange(len(pcm)), pcm).astype(np.float32)
sample_rate = 24000
encoder = opuslib.Encoder(sample_rate, 1, opuslib.APPLICATION_VOIP)
frame_size = sample_rate * frame_ms // 1000 # samples per frame
serial = 1
seq = 0
# --- BOS page: OpusHead ---
opus_head = struct.pack("<8sBBHIhB", b"OpusHead", 1, 1, 312, 24000, 0, 0)
pages = _ogg_page(serial, 0, seq, bos=True, eos=False, opus_head)
seq += 1
# --- Comment page: OpusTags ---
vendor = b"PolarGrid"
opus_tags = struct.pack("<8sI", b"OpusTags", len(vendor)) + vendor + struct.pack("= len(pcm_i16)
pages += _ogg_page(serial, granule, seq, bos=False, eos=is_last, opus_frame)
seq += 1
pos += frame_size
return pages
# ---------------------------------------------------------------------------
# Main client
# ---------------------------------------------------------------------------
async def run(api_key: str, audio_path: str, region: str, voice: str,
persona: str, output_path: str):
# Load and prepare audio
pcm, sr = sf.read(audio_path, dtype="float32", always_2d=True)
pcm = pcm[:, 0] # mono
print(f"Loaded {audio_path}: {len(pcm)/sr:.1f}s at {sr} Hz")
ogg_data = encode_audio_to_ogg_opus(pcm, sample_rate=sr)
print(f"Encoded to Ogg/Opus: {len(ogg_data)} bytes")
# Build WebSocket URL
from urllib.parse import quote
url = (
f"wss://api.{region}.edge.polargrid.ai/v1/voice/personaplex"
f"?voice={voice}&persona={quote(persona)}&token={api_key}"
)
response_audio = bytearray()
transcript_parts = []
# Connect — disable ping to avoid teardown (moshi does not pong)
async with websockets.connect(url, ping_interval=None,
max_size=None) as ws:
# Step 1: Wait for the handshake (0x00)
print("Waiting for handshake...")
msg = await ws.recv()
if isinstance(msg, bytes) and msg[0] == TAG_HANDSHAKE:
print("Handshake received — session is ready")
else:
raise RuntimeError(f"Expected handshake, got: {msg[:20]!r}")
# Step 2: Send audio frames with 0x01 tag prefix
CHUNK = 4096
for i in range(0, len(ogg_data), CHUNK):
frame = bytes([TAG_AUDIO]) + ogg_data[i : i + CHUNK]
await ws.send(frame)
print(f"Sent {len(ogg_data)} bytes of audio")
# Step 3: Send end-of-stream control frame
await ws.send(bytes([TAG_CONTROL]) + b"eos")
print("Sent eos — waiting for response...")
# Step 4: Receive response audio and transcript
try:
async for msg in ws:
if not isinstance(msg, bytes) or len(msg) < 1:
continue
tag = msg[0]
payload = msg[1:]
if tag == TAG_AUDIO:
response_audio.extend(payload)
elif tag == TAG_TEXT:
text = payload.decode("utf-8")
transcript_parts.append(text)
print(f" transcript: {text}")
except websockets.exceptions.ConnectionClosed:
pass
# Save response audio (raw Opus/Ogg — playable with ffplay, mpv, VLC)
if response_audio:
with open(output_path, "wb") as f:
f.write(response_audio)
print(f"\nSaved response audio to {output_path} ({len(response_audio)} bytes)")
if transcript_parts:
print(f"Full transcript: {''.join(transcript_parts)}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="PersonaPlex client example")
parser.add_argument("--api-key", required=True, help="Your pg_* API key")
parser.add_argument("--audio", required=True, help="Path to input audio file (WAV)")
parser.add_argument("--region", default="yto-01", help="Edge region (default: yto-01)")
parser.add_argument("--voice", default="NATF0", help="Voice ID (default: NATF0)")
parser.add_argument("--persona", default="A helpful voice assistant.",
help="Persona prompt")
parser.add_argument("--output", default="response.ogg", help="Output file path")
args = parser.parse_args()
asyncio.run(run(args.api_key, args.audio, args.region, args.voice,
args.persona, args.output))
```
**Usage:**
```bash theme={null}
python personaplex_client.py \
--api-key pg_your_key_here \
--audio question.wav \
--persona "A friendly tour guide for Vancouver."
# => response.ogg (playable with ffplay, mpv, or VLC)
```
### Node.js
```bash theme={null}
npm install ws
```
```javascript theme={null}
#!/usr/bin/env node
/**
* Send a pre-encoded Ogg/Opus file to PersonaPlex and save the response.
*
* This example reads a file that is already in Ogg/Opus format (mono, 24 kHz).
* To convert a WAV to the right format:
* ffmpeg -i input.wav -ac 1 -ar 24000 -c:a libopus input.ogg
*/
const { readFileSync, writeFileSync } = require("fs");
const WebSocket = require("ws");
// ---------------------------------------------------------------------------
// Wire protocol tags
// ---------------------------------------------------------------------------
const TAG_HANDSHAKE = 0x00; // Server → Client: session ready
const TAG_AUDIO = 0x01; // Both directions: Opus audio (Ogg container)
const TAG_TEXT = 0x02; // Both directions: UTF-8 text
const TAG_CONTROL = 0x03; // Client → Server: stream boundary (eos)
// ---------------------------------------------------------------------------
// Config — edit these or pass via environment variables
// ---------------------------------------------------------------------------
const API_KEY = process.env.POLARGRID_API_KEY || "pg_your_key_here";
const INPUT = process.argv[2] || "input.ogg"; // pre-encoded Ogg/Opus file
const REGION = process.env.POLARGRID_REGION || "yto-01";
const VOICE = process.env.POLARGRID_VOICE || "NATF0";
const PERSONA = process.env.POLARGRID_PERSONA || "A helpful voice assistant.";
const OUTPUT = process.argv[3] || "response.ogg";
// ---------------------------------------------------------------------------
// Main
// ---------------------------------------------------------------------------
const oggData = readFileSync(INPUT);
console.log(`Loaded ${INPUT}: ${oggData.length} bytes`);
const url = new URL(`wss://api.${REGION}.edge.polargrid.ai/v1/voice/personaplex`);
url.searchParams.set("voice", VOICE);
url.searchParams.set("persona", PERSONA);
url.searchParams.set("token", API_KEY);
// Disable automatic ping — moshi does not respond to pongs
const ws = new WebSocket(url.toString(), { pingInterval: 0 });
ws.binaryType = "arraybuffer";
const responseChunks = [];
const transcriptParts = [];
ws.on("open", () => {
console.log("WebSocket connected, waiting for handshake...");
});
ws.on("message", (data) => {
const buf = Buffer.from(data);
if (buf.length < 1) return;
const tag = buf[0];
const payload = buf.subarray(1);
if (tag === TAG_HANDSHAKE) {
// Step 1: Handshake received — safe to send audio
console.log("Handshake received — sending audio...");
// Step 2: Send audio in chunks with 0x01 tag prefix
const CHUNK = 4096;
for (let i = 0; i < oggData.length; i += CHUNK) {
const slice = oggData.subarray(i, i + CHUNK);
const frame = Buffer.concat([Buffer.from([TAG_AUDIO]), slice]);
ws.send(frame);
}
console.log(`Sent ${oggData.length} bytes of audio`);
// Step 3: Send end-of-stream control frame
ws.send(Buffer.concat([Buffer.from([TAG_CONTROL]), Buffer.from("eos")]));
console.log("Sent eos — waiting for response...");
} else if (tag === TAG_AUDIO) {
responseChunks.push(payload);
} else if (tag === TAG_TEXT) {
const text = payload.toString("utf-8");
transcriptParts.push(text);
console.log(` transcript: ${text}`);
}
});
ws.on("close", () => {
// Step 4: Save response audio
if (responseChunks.length > 0) {
const responseAudio = Buffer.concat(responseChunks);
writeFileSync(OUTPUT, responseAudio);
console.log(`\nSaved response audio to ${OUTPUT} (${responseAudio.length} bytes)`);
}
if (transcriptParts.length > 0) {
console.log(`Full transcript: ${transcriptParts.join("")}`);
}
});
ws.on("error", (err) => {
console.error("WebSocket error:", err.message);
process.exit(1);
});
```
**Usage:**
```bash theme={null}
# First, convert your audio to Ogg/Opus mono 24 kHz:
ffmpeg -i question.wav -ac 1 -ar 24000 -c:a libopus question.ogg
# Run the client:
POLARGRID_API_KEY=pg_your_key_here node personaplex_client.js question.ogg
# => response.ogg
```
The Node.js example reads a pre-encoded Ogg/Opus file to keep the code short. If you need to encode from WAV at runtime, use `ffmpeg` as a subprocess or the `@discordjs/opus` package to encode PCM frames, then wrap them in Ogg pages (see the Python example for the page structure).
# Rate Limits
Source: https://polargrid.mintlify.app/guides/rate-limits
Understand PolarGrid's rate limits and how to handle them in production
# Rate Limits
PolarGrid enforces rate limits to ensure fair usage and platform stability. These limits apply uniformly across all inference endpoints (LLM, STT, TTS) and all models.
## Current Limits
| Limit | Value | Scope |
| ------------------- | ----- | ------------------------------------------------------------------------------------ |
| Requests per minute | 100 | Per user, counted independently on each edge node (per client IP if unauthenticated) |
* **Request rate limit**: A per-minute window allows up to 100 requests per user. The limit is keyed to the user who owns the API key, so every key created by the same user draws from one shared 100/min budget. Each edge node keeps its own counter and resets it at the minute boundary, so traffic spread across several nodes gets a separate budget on each (see [Distribute requests across edges](#distribute-requests-across-edges)). If no API key is provided, the limit applies per client IP address.
This limit applies identically to all endpoints: `/v1/chat/completions`, `/v1/completions`, `/v1/audio/speech`, and `/v1/audio/transcriptions`.
## What Happens When You Hit a Limit
When the limit is exceeded, the gateway returns an **HTTP 429 Too Many Requests** response:
```json theme={null}
{
"error": {
"message": "Rate limit exceeded",
"type": "rate_limit_error",
"code": "rate_limit_exceeded"
}
}
```
The SDK surfaces this as a `RateLimitError`. When the response carries a `Retry-After` header, the SDK exposes its value as `retryAfter` (in seconds); when the header is absent, `retryAfter` is `undefined`, so apply your own backoff (see below). See [Error Handling](/guides/error-handling) for the full error type reference.
## Retry Strategy
Use **exponential backoff with jitter** to avoid thundering-herd problems when retrying after a 429:
```javascript JavaScript theme={null}
async function withBackoff(fn, maxRetries = 5) {
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
return await fn();
} catch (error) {
if (error.status !== 429 || attempt === maxRetries - 1) {
throw error;
}
const baseDelay = Math.pow(2, attempt) * 1000; // 1s, 2s, 4s, 8s, 16s
const jitter = Math.random() * 1000;
await new Promise(r => setTimeout(r, baseDelay + jitter));
}
}
}
const response = await withBackoff(() =>
client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Hello' }],
})
);
```
```python Python theme={null}
import asyncio
import random
async def with_backoff(fn, max_retries=5):
for attempt in range(max_retries):
try:
return await fn()
except Exception as e:
if getattr(e, "status", None) != 429 or attempt == max_retries - 1:
raise
base_delay = (2 ** attempt) # 1s, 2s, 4s, 8s, 16s
jitter = random.uniform(0, 1)
await asyncio.sleep(base_delay + jitter)
response = await with_backoff(lambda: client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello"}],
}))
```
The PolarGrid SDKs include built-in retry with exponential backoff for transient errors (including 429s). Configure via `maxRetries` when initializing the client. The examples above are for custom retry logic beyond the defaults.
## Best Practices for Production
### Distribute requests across edges
Use the [autorouter](/guides/regions) to spread traffic across multiple edge nodes. Each edge node maintains its own rate limit counters, so distributing requests reduces the chance of hitting limits on any single node.
```javascript theme={null}
const client = new PolarGrid({
apiKey: 'pg_...',
// Autorouter selects the nearest available edge
baseUrl: 'https://autorouter.polargrid.ai',
});
```
### Implement client-side throttling
Rather than relying on server-side 429 responses, proactively throttle requests in your application:
```javascript theme={null}
// Simple token bucket rate limiter
class RateLimiter {
constructor(maxRequests = 90, windowMs = 60_000) {
this.tokens = maxRequests;
this.maxTokens = maxRequests;
this.windowMs = windowMs;
this.lastRefill = Date.now();
}
async acquire() {
this.refill();
if (this.tokens <= 0) {
const waitMs = this.windowMs - (Date.now() - this.lastRefill);
await new Promise(r => setTimeout(r, waitMs));
this.refill();
}
this.tokens--;
}
refill() {
const now = Date.now();
if (now - this.lastRefill >= this.windowMs) {
this.tokens = this.maxTokens;
this.lastRefill = now;
}
}
}
const limiter = new RateLimiter(90); // Stay under the 100/min limit
async function safeRequest(client, params) {
await limiter.acquire();
return client.chatCompletion(params);
}
```
### Use streaming to reduce request count
A single streaming request holds one connection open while tokens are generated, rather than making multiple polling requests. This is especially effective for long responses.
```javascript theme={null}
// One request, streamed over a single connection
for await (const chunk of client.chatCompletionStream({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Write a detailed analysis...' }],
})) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
```
See the [Streaming guide](/guides/streaming) for full details.
### Batch where possible
If you have multiple independent prompts, send them as separate requests but pace them to stay within your rate limit. Avoid firing all requests simultaneously.
## Custom Limits
Enterprise customers can request custom rate limits tailored to their workload. Contact [support@polargrid.ai](mailto:support@polargrid.ai) or reach out to your account representative to discuss higher limits.
# Regions
Source: https://polargrid.mintlify.app/guides/regions
Understanding PolarGrid edge regions
# Regions
PolarGrid runs GPU infrastructure at edge locations to minimize latency.
## Available Regions
| ID | Name | Location | Country |
| -------- | ------------- | -------------- | ------- |
| `yto-01` | Toronto | Canada Central | CA |
| `yul-01` | Montreal | Canada East | CA |
| `yvr-02` | Vancouver | Canada West | CA |
| `nyc-01` | New York | US East | US |
| `nyc-02` | New York 02 | US East | US |
| `dfw-01` | Dallas | US Central | US |
| `dfw-02` | Dallas 02 | US Central | US |
| `sfo-01` | San Francisco | US West | US |
| `lax-01` | Los Angeles | US West | US |
| `sea-01` | Seattle | US West | US |
| `chi-01` | Chicago | US Central | US |
| `phx-01` | Phoenix | US West | US |
| `was-01` | Washington DC | US East | US |
| `mia-01` | Miami | US East | US |
| `sfo-03` | San Francisco | US West | US |
## Endpoint URLs
```
https://api.{region-id}.edge.polargrid.ai
```
Examples:
* `https://api.yto-01.edge.polargrid.ai`
* `https://api.yul-01.edge.polargrid.ai`
* `https://api.yvr-02.edge.polargrid.ai`
* `https://api.nyc-01.edge.polargrid.ai`
* `https://api.dfw-01.edge.polargrid.ai`
* `https://api.sfo-01.edge.polargrid.ai`
* `https://api.lax-01.edge.polargrid.ai`
* `https://api.sea-01.edge.polargrid.ai`
* `https://api.chi-01.edge.polargrid.ai`
* `https://api.phx-01.edge.polargrid.ai`
* `https://api.was-01.edge.polargrid.ai`
* `https://api.mia-01.edge.polargrid.ai`
* `https://api.sfo-03.edge.polargrid.ai`
## Auto-Routing
The SDKs can automatically select the fastest region:
```javascript JavaScript theme={null}
// Calls the autorouter which returns the optimal edge based on your origin
const client = await PolarGrid.create({
apiKey: "pg_your_api_key",
debug: true, // See selected region
});
// [PolarGrid] Auto-routing: selected Toronto (yto-01)
console.log(client.getRegionId()); // 'yto-01'
console.log(client.getRegionName()); // 'Toronto'
```
```python Python theme={null}
# Calls the autorouter which returns the optimal edge based on your origin
client = await PolarGrid.create(api_key="pg_your_api_key", debug=True)
# [PolarGrid] Auto-routing: selected Toronto (yto-01)
print(client.get_region_id()) # 'yto-01'
print(client.get_region_name()) # 'Toronto'
```
## Explicit Region Selection
You can specify a region by ID or alias:
```javascript JavaScript theme={null}
// By alias (case-insensitive)
const client = new PolarGrid({
apiKey: "pg_...",
region: "toronto", // or 'vancouver', 'montreal'
});
// By ID
const client = new PolarGrid({
apiKey: "pg_...",
region: "yto-01", // or 'yvr-02', 'yul-01'
});
```
```python Python theme={null}
# By alias
client = PolarGrid(api_key="pg_...", region="toronto")
# By ID
client = PolarGrid(api_key="pg_...", region="yto-01")
```
## Region Aliases
For convenience, these aliases are supported:
| Alias | Region ID |
| -------------------------------------------- | --------- |
| `toronto`, `yto` | `yto-01` |
| `montreal`, `yul` | `yul-01` |
| `vancouver`, `yvr` | `yvr-02` |
| `new-york`, `newyork`, `nyc` | `nyc-01` |
| `dallas`, `dfw` | `dfw-01` |
| `san-francisco`, `sanfrancisco`, `sf`, `sfo` | `sfo-01` |
The `-02` variants (e.g. `nyc-02`, `dfw-02`) and `lax-01`, `sea-01`, `chi-01`, `phx-01`, `was-01`, `mia-01`, `sfo-03` are reachable only by explicit region ID — they have no alias.
## Checking Latency
### CLI
```bash theme={null}
# List regions with current latency
polargrid regions list
# Detailed ping test
polargrid regions ping --count 5
```
### SDK
Auto-routing logs latency when debug is enabled:
```javascript theme={null}
const client = await PolarGrid.create({
apiKey: "pg_...",
debug: true,
});
// [PolarGrid] Auto-routing: selected Toronto (yto-01)
```
## Default Region
If you don't specify a region and don't use auto-routing, the SDKs default to **Toronto** (`yto-01`).
For CLI, you can set a default:
```bash theme={null}
polargrid config set default_region yvr-02
```
## Health Checks and Fallback
### Checking Region Health
Each edge node exposes a `/health` endpoint:
```bash theme={null}
curl https://api.yto-01.edge.polargrid.ai/health
```
A healthy response includes the node ID, runtime info, and loaded models. Use this to verify a region is operational before pinning traffic to it.
### Autorouter Discovery
The autorouter returns the single best edge for the caller. By default it
detects the caller's country (from request geo) and returns the **nearest edge
in that country** — a US caller is routed to the nearest US edge, a Canadian
caller to the nearest Canadian edge — even when an edge across the border is
physically closer. If no healthy edge exists in the caller's country, it falls
back to the globally nearest edge.
```bash theme={null}
curl https://autorouter.polargrid.ai/v1/route
# → {"region":"yto-01","name":"Toronto Edge","endpoint":"https://api.yto-01.edge.polargrid.ai:443","ttl":3600,"scope":"country"}
```
The `endpoint` field is the base URL you POST inference to. The `ttl` (seconds) is a hint for how long the SDK should cache the choice before re-asking.
#### Choosing the routing scope
Country affinity is the default because it keeps traffic domestic, which is
usually what data-residency-sensitive workloads want. When raw proximity
matters more than staying in-country, pass `scope=global` to let the router
cross borders:
| `scope` | Behavior |
| --------------------- | ---------------------------------------------------------------------------------------------- |
| `country` *(default)* | Nearest edge in the caller's own country; crosses the border only if that country has no edge. |
| `global` | Nearest edge worldwide, ignoring borders. |
```bash theme={null}
# A Detroit caller: country scope stays in the US, global scope takes Toronto.
curl "https://autorouter.polargrid.ai/v1/route?scope=global"
# → {"region":"yto-01",...,"scope":"global"}
```
Unrecognized values fall back to `country`, so a typo can never silently widen
routing to another country. The response echoes the `scope` that was actually
applied — check it if you need to confirm your request was honored rather than
silently defaulted.
`scope=global` opts out of country affinity **and** the network-aware
on-ramp preference applied to callers whose country has no edge. It ranks
purely by distance plus edge queue depth — the same load term the default
uses — with no network-topology awareness. Great-circle distance is not
always the lowest-latency choice across oceans, so benchmark before
switching long-haul traffic to `global`.
Both SDKs expose this as a client option:
```javascript theme={null}
const client = await PolarGrid.create({ apiKey, routingScope: 'global' });
```
```python theme={null}
client = await PolarGrid.create(api_key=api_key, routing_scope="global")
```
#### Routing to a warm model
Discovery can also narrow candidates to edges that already have a specific
model loaded, so your first request hits a warm node instead of paying a
dynamic model load. Model availability is a hard filter; scope then picks the
nearest among the nodes that qualify — the two options compose:
```javascript theme={null}
const client = await PolarGrid.create({
apiKey,
routingModel: 'kokoro', // only edges with kokoro READY
routingScope: 'global', // nearest such edge worldwide
});
```
```python theme={null}
client = await PolarGrid.create(
api_key=api_key,
routing_model="kokoro",
routing_scope="global",
)
```
```bash theme={null}
curl "https://autorouter.polargrid.ai/v1/route?model=kokoro&scope=global"
```
If no edge currently serves the model, the raw endpoint returns **404**; the
SDKs catch that and fall back to the default edge (uncached — the miss is
transient) so connecting still succeeds. The first request then triggers a
dynamic model load instead of hitting a warm node.
These options are ignored when you pin a `region` or `baseUrl` — those bypass
the autorouter entirely. Each scope/model combination is cached separately, so
switching either re-asks the router rather than reusing another combination's
answer.
### Fallback Strategy
If you're pinning to a specific region (not using auto-routing), we recommend this fallback pattern:
1. **Primary**: Your chosen region (e.g., `yto-01`)
2. **Fallback**: Try the next-closest region if the primary returns 5xx or times out
3. **Auto-route**: Fall back to `PolarGrid.create()` which calls the autorouter
```javascript theme={null}
async function createWithFallback(apiKey, preferredRegion, fallbackRegion) {
try {
const client = new PolarGrid({ apiKey, region: preferredRegion });
await client.listModels(); // verify it's reachable
return client;
} catch {
console.warn(`${preferredRegion} unreachable, trying ${fallbackRegion}`);
try {
const fallback = new PolarGrid({ apiKey, region: fallbackRegion });
await fallback.listModels(); // verify fallback is reachable too
return fallback;
} catch {
console.warn('Fallback failed, using auto-routing');
return PolarGrid.create({ apiKey });
}
}
}
```
For most use cases, `PolarGrid.create()` is the best option — it handles region selection and failover automatically. Only pin to a specific region if you need deterministic routing for compliance or latency guarantees.
## Direct API Access
For raw HTTP requests, use the full endpoint:
```bash theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"model": "qwen-3.8-27b", "messages": [...]}'
```
# Streaming
Source: https://polargrid.mintlify.app/guides/streaming
Stream tokens as they're generated
# Streaming
Stream response tokens in real-time for better user experience.
## Why Stream?
Without streaming, users wait for the entire response before seeing anything. With streaming, tokens appear as they're generated — critical for interactive applications.
## Chat Completion Streaming
```javascript JavaScript theme={null}
for await (const chunk of client.chatCompletionStream({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Tell me a story about a robot' }],
maxTokens: 500,
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) {
process.stdout.write(content);
}
// Check for completion
if (chunk.choices[0]?.finishReason) {
console.log('\n\nFinished:', chunk.choices[0].finishReason);
}
}
```
```python Python theme={null}
async for chunk in client.chat_completion_stream({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Tell me a story about a robot"}],
"max_tokens": 500,
}):
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
# Check for completion
if chunk.choices[0].finish_reason:
print(f"\n\nFinished: {chunk.choices[0].finish_reason}")
```
## Text Completion Streaming
```javascript JavaScript theme={null}
for await (const chunk of client.completionStream({
prompt: 'Once upon a time',
model: 'qwen-3.8-27b',
maxTokens: 200,
})) {
process.stdout.write(chunk.choices[0].text);
}
```
```python Python theme={null}
async for chunk in client.completion_stream({
"prompt": "Once upon a time",
"model": "qwen-3.8-27b",
"max_tokens": 200,
}):
print(chunk.choices[0].text, end="", flush=True)
```
## Chunk Format
Each streaming chunk contains a delta (incremental change):
```json theme={null}
{
"id": "chatcmpl-abc123",
"object": "chat.completion.chunk",
"created": 1234567890,
"model": "qwen-3.8-27b",
"choices": [
{
"index": 0,
"delta": {
"role": "assistant", // Only in first chunk
"content": "Once" // Token content
},
"finish_reason": null // null until final chunk
}
]
}
```
## SSE Format (Raw API)
When using the API directly with `stream: true`, responses use Server-Sent Events:
```
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":"Once"}}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":" upon"}}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
## Browser Example
```javascript theme={null}
async function streamChat(prompt) {
const outputElement = document.getElementById('output');
outputElement.textContent = '';
for await (const chunk of client.chatCompletionStream({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: prompt }],
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) {
outputElement.textContent += content;
}
}
}
```
## React Hook Example
```typescript theme={null}
import { useState, useCallback } from 'react';
function useChatStream(client: PolarGrid) {
const [response, setResponse] = useState('');
const [isStreaming, setIsStreaming] = useState(false);
const stream = useCallback(async (messages: Message[]) => {
setResponse('');
setIsStreaming(true);
try {
for await (const chunk of client.chatCompletionStream({
model: 'qwen-3.8-27b',
messages,
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) {
setResponse(prev => prev + content);
}
}
} finally {
setIsStreaming(false);
}
}, [client]);
return { response, isStreaming, stream };
}
```
## Finish Reasons
| Reason | Description |
| ---------------- | --------------------------------------- |
| `stop` | Natural completion or stop sequence hit |
| `length` | Max tokens reached |
| `content_filter` | Content was filtered |
| `tool_calls` | Model is calling a tool/function |
## Tips
1. **Always check for content**: Some chunks may have empty content
2. **Handle the finish reason**: Know why generation stopped
3. **Buffer if needed**: For sentence-by-sentence display, buffer until punctuation
4. **Error handling**: Wrap in try/catch for network errors mid-stream
# Troubleshooting
Source: https://polargrid.mintlify.app/guides/troubleshooting
Common errors and how to fix them
# Troubleshooting
Quick reference for the most common errors you'll hit when integrating with PolarGrid.
## HTTP Error Reference
| Code | Error | Cause | Fix |
| ---- | ------------------------------------ | --------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 401 | `Invalid token: Not enough segments` | Malformed token sent to a non-edge endpoint expecting a JWT | Edge endpoints accept `pg_*` keys directly. For auth-service endpoints, exchange your key via `POST https://auth.polargrid.ai/v1/auth/session`. See [Authentication](/authentication). |
| 401 | `Token has expired` | Your session JWT has expired (default TTL: 1 hour) | Request a new session token, or use direct API key auth (no JWT needed on edge endpoints). |
| 403 | `{"Message": null}` | Request blocked at the AWS infrastructure level (CloudFront/WAF) | Verify your API key is active in the [dashboard](https://app.polargrid.ai/dashboard/settings). This error comes from AWS infrastructure, not the PolarGrid application. |
| 402 | `Billing access denied` | Your organization has no payment method or credits | Add a payment method in Settings > Billing, or contact support for credits. |
| 404 | `Model not loaded` | The model you requested is not deployed on this edge node | Call `GET /v1/models` to see available models, or switch to a different region. |
| 405 | `Method Not Allowed` | You sent an unsupported HTTP method (e.g., POST/PUT to a GET-only endpoint) | Check the [API Reference](/api-reference/overview) for the correct HTTP method for each endpoint. |
| 429 | `Rate limit exceeded` | Too many requests in a short window | Back off and retry. Check `Retry-After` header for the wait time. |
| 502 | `TTS backend error` | The TTS synthesis engine failed (often bad voice ID or empty input) | Check that your `voice` parameter matches a valid voice ID and `input` is not empty. |
## Authentication Issues
### "I'm sending my API key but getting 401"
Edge endpoints accept `pg_*` API keys directly — no token exchange needed:
```bash theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/models \
-H "Authorization: Bearer pg_your_key_here"
```
The SDK also sends `pg_*` keys directly. If you're still getting 401:
* **Check the key is active** in Settings > API Keys (not revoked)
* **Check the endpoint** — some non-edge endpoints (e.g., management APIs) may require a session JWT
* **Check the format** — the key must start with `pg_`
For endpoints that require a session JWT, exchange your API key first:
```bash theme={null}
TOKEN=$(curl -s -X POST https://auth.polargrid.ai/v1/auth/session \
-H "Authorization: Bearer pg_your_key_here" \
-H "Content-Type: application/json" \
| jq -r '.session_token')
curl https://api.yto-01.edge.polargrid.ai/v1/models \
-H "Authorization: Bearer $TOKEN"
```
### "My key works in the playground but not in my code"
The playground uses the same auth flow as the SDK. Common causes:
* **Wrong base URL**: Use regional edge URLs like `api.yto-01.edge.polargrid.ai`, not `api.polargrid.ai`
* **Missing Content-Type**: POST requests need `Content-Type: application/json`
* **Key permissions**: Check that your key has `read-write` permissions in the dashboard
## Region and Routing Issues
### "Which region should I use?"
Use the autorouter for automatic region selection:
```bash theme={null}
curl https://autorouter.polargrid.ai/v1/route
```
Or pick a specific region:
| Region | Endpoint |
| --------- | ------------------------------ |
| Toronto | `api.yto-01.edge.polargrid.ai` |
| Vancouver | `api.yvr-02.edge.polargrid.ai` |
| Montreal | `api.yul-01.edge.polargrid.ai` |
### "I'm getting high latency"
1. Check you're hitting the nearest region (use the autorouter)
2. First requests may be slower due to model cold start
3. Use `GET /health` to check if the edge node is healthy before sending inference requests
## Browser / Frontend Issues
### "ReferenceError: process is not defined"
The SDK references `process.env` for configuration fallbacks. In browser environments (Vite, esbuild, Parcel), `process` is not defined.
**Fix:** Pass all configuration explicitly at init:
```javascript theme={null}
const client = new PolarGrid({
apiKey: 'pg_your_key',
baseUrl: 'https://api.yto-01.edge.polargrid.ai',
});
```
### "CORS error when calling the API from my frontend"
Edge inference endpoints include `Access-Control-Allow-Origin: *` via CORS middleware, so browser-origin requests work directly. If you're seeing CORS errors:
1. **Check the URL**: Make sure you're hitting a regional edge URL (`api.yto-01.edge.polargrid.ai`), not a misconfigured proxy
2. **Check the auth endpoint**: If the CORS error is on `auth.polargrid.ai`, ensure your auth-service deployment includes CORS headers (added in POL-230)
3. **Check preflight**: `OPTIONS` requests must return 2xx with CORS headers — if your proxy strips them, the browser blocks the real request
## TTS Issues
### "TTS returns 0 bytes or empty audio"
Common causes:
* Empty `input` field
* Invalid `voice` parameter (check available voices in the [TTS docs](/api-reference/text-to-speech))
* Quoted text with special characters can sometimes cause issues
### "TTS generation time is very slow"
TTS generation time scales linearly with input length for batch requests. For long text, consider:
* Breaking text into shorter segments
* Using streaming TTS (`stream: true`) for faster time-to-first-audio
## Still Stuck?
* Check the [API Reference](/api-reference/overview) for endpoint details
* Review the [Error Handling](/guides/error-handling) guide for SDK error types
* Contact support at [support@polargrid.ai](mailto:support@polargrid.ai)
# Voice AI
Source: https://polargrid.mintlify.app/guides/voice
Text-to-speech and speech-to-text
# Voice AI
PolarGrid provides low-latency voice capabilities at the edge.
## Text-to-Speech (TTS)
Convert text to natural-sounding speech.
### Basic Usage
```javascript JavaScript theme={null}
const audioBuffer = await client.textToSpeech({
model: 'kokoro-82m',
input: 'Hello from PolarGrid!',
voice: 'af_bella',
responseFormat: 'wav',
});
// Fully-formed RIFF/WAVE container — playable directly.
import { writeFile } from 'fs/promises';
await writeFile('speech.wav', Buffer.from(audioBuffer));
```
```python Python theme={null}
audio_bytes = await client.text_to_speech({
"model": "kokoro-82m",
"input": "Hello from PolarGrid!",
"voice": "af_bella",
"response_format": "wav",
})
with open("speech.wav", "wb") as f:
f.write(audio_bytes)
```
### Voices
The `kokoro-82m` model exposes eight voices across American and British English:
| Voice ID | Accent / gender |
| ------------- | ------------------------ |
| `af_bella` | American English, female |
| `af_sarah` | American English, female |
| `am_adam` | American English, male |
| `am_michael` | American English, male |
| `bf_emma` | British English, female |
| `bf_isabella` | British English, female |
| `bm_george` | British English, male |
| `bm_lewis` | British English, male |
Kokoro-82M itself ships many more voices (additional English tiers plus Japanese, Mandarin, Spanish, French, Hindi, Italian, and Brazilian Portuguese) — see the upstream [Kokoro-82M VOICES.md](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/VOICES.md). Only the eight above are exposed through the PolarGrid SDKs today.
### Speed Control
Adjust playback speed from 0.25x to 4.0x:
```javascript theme={null}
const audioBuffer = await client.textToSpeech({
model: 'kokoro-82m',
input: 'This will be spoken slowly.',
voice: 'af_bella',
speed: 0.75, // Slower
});
```
### Audio Format
Audio is generated at **24 kHz, 16-bit, mono**. The container is selected via `response_format`:
| `response_format` | Content-Type | Streaming? | Use when |
| ----------------- | ------------ | ---------- | ------------------------------------------------------------------------------ |
| `pcm` *(default)* | `audio/pcm` | Yes | Real-time voice-agent pipelines — lowest first-byte latency. |
| `wav` | `audio/wav` | No | You need a playable file with no client-side post-processing. |
| `mp3` | `audio/mpeg` | No | Bandwidth matters; bytes are encoded server-side via `libmp3lame` at 128 kbps. |
The default differs from OpenAI's `/v1/audio/speech` (which defaults to `mp3`). PolarGrid defaults to `pcm` to keep streaming TTS first-byte latency minimal — the PolarGrid SDKs default to `mp3` for OpenAI-style behavior end-to-end, so pass `responseFormat` / `response_format` explicitly when calling via the SDK.
`opus`, `aac`, and `flac` from the OpenAI spec are not yet supported in batch mode — requesting them returns `400`. For streaming, `opus` is supported (see below); for batch, transcode PCM client-side if you need one of those:
```bash theme={null}
ffmpeg -f s16le -ar 24000 -ac 1 -i speech.pcm speech.opus
```
### Streaming
For real-time playback or voice-agent pipelines, set `stream: true` and use `response_format: 'pcm'` (lowest latency) or `'opus'` (compressed). See the [TTS API reference](/api-reference/text-to-speech#streaming) for the full contract and the formats / models matrix.
```javascript JavaScript theme={null}
for await (const chunk of client.textToSpeechStream({
model: 'kokoro-82m',
input: 'Streaming hello',
voice: 'af_bella',
responseFormat: 'opus',
})) {
audioPlayer.appendChunk(chunk);
}
```
```python Python theme={null}
async for chunk in client.text_to_speech_stream({
"model": "kokoro-82m",
"input": "Streaming hello",
"voice": "af_bella",
"response_format": "pcm",
}):
voice_agent_track.send_pcm(chunk)
```
Streaming `wav` or `mp3` returns `400`; transcode client-side from `pcm` if you need a different container. For `tada-3b-ml`, streaming does not honor the `speed` parameter — use `speed=1.0` (or omit it).
### Raw HTTP Contract
If you're not using the SDK, here's the full request/response shape:
```bash Batch Request theme={null}
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer pg_your_key" \
-H "Content-Type: application/json" \
-d '{
"model": "kokoro-82m",
"input": "Hello from PolarGrid!",
"voice": "af_bella",
"response_format": "wav",
"speed": 1.0
}' \
--output speech.wav
```
```bash Streaming Request theme={null}
curl -X POST https://api.yto-01.edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer pg_your_key" \
-H "Content-Type: application/json" \
-d '{
"model": "kokoro-82m",
"input": "Streaming hello from PolarGrid!",
"voice": "af_bella",
"response_format": "pcm",
"stream": true
}' \
--output speech.pcm
```
**Batch response:** Binary audio in the requested container format. `Content-Type` matches the format (`audio/wav`, `audio/pcm`).
**Streaming response:** Chunked transfer-encoding with raw audio bytes. Headers include `X-Polargrid-Stream: 1` and `X-Polargrid-Sample-Rate: 24000`. PCM is 16-bit signed little-endian mono at 24 kHz.
## Speech-to-Text (STT)
Transcribe audio to text.
The `file` parameter accepts `File | Blob` in JavaScript, and `Path` or any file-like object in Python. Buffers, path strings, and streams are not accepted directly — wrap them in a `Blob` or `File` first.
### Basic Transcription
```javascript JavaScript theme={null}
const file = new File([audioData], 'recording.mp3', { type: 'audio/mpeg' });
const result = await client.transcribe({
file,
model: 'whisper-large-v3-turbo',
language: 'en', // Optional: hint the language
});
console.log(result.text);
```
```python Python theme={null}
from pathlib import Path
result = await client.transcribe(
file=Path("recording.mp3"),
request={
"model": "whisper-large-v3-turbo",
"language": "en",
}
)
print(result.text)
```
### Verbose Output with Timestamps
Get word-level timestamps:
```javascript JavaScript theme={null}
const result = await client.transcribe({
file,
model: 'whisper-large-v3-turbo',
responseFormat: 'verbose_json',
});
console.log(`Duration: ${result.duration}s`);
console.log(`Language: ${result.language}`);
result.segments.forEach(segment => {
console.log(`[${segment.start.toFixed(2)} - ${segment.end.toFixed(2)}] ${segment.text}`);
});
```
```python Python theme={null}
result = await client.transcribe(
file=audio_file,
request={
"model": "whisper-large-v3-turbo",
"response_format": "verbose_json",
}
)
print(f"Duration: {result.duration}s")
print(f"Language: {result.language}")
for segment in result.segments:
print(f"[{segment.start:.2f} - {segment.end:.2f}] {segment.text}")
```
### Subtitle Formats
Generate subtitles directly:
```javascript theme={null}
// SRT format
const srt = await client.transcribe({
file,
model: 'whisper-large-v3-turbo',
responseFormat: 'srt',
});
// WebVTT format
const vtt = await client.transcribe({
file,
model: 'whisper-large-v3-turbo',
responseFormat: 'vtt',
});
```
## Voice Chat (Request/Response Loop)
Transcription in this loop requires a completed audio file — the user must finish speaking before the request is sent. For streaming realtime audio, see [PersonaPlex](/guides/personaplex) (multi-modal, single model) or the [Modular Pipeline Agent](/guides/voice-agent) (STT → LLM → TTS with streaming events).
Combine TTS and STT for voice conversations. This example records a short utterance from the microphone, transcribes it, passes the text through the chat model, and speaks the response:
```javascript theme={null}
// Browser: capture a Blob from the microphone via MediaRecorder,
// then run it through transcribe → chat → TTS.
async function voiceChat(audioBlob) {
// audioBlob: a Blob produced by MediaRecorder, e.g.
// const recorder = new MediaRecorder(stream);
// recorder.ondataavailable = (e) => chunks.push(e.data);
// const audioBlob = new Blob(chunks, { type: 'audio/webm' });
// 1. Transcribe user speech
const transcription = await client.transcribe({
file: audioBlob,
model: 'whisper-large-v3-turbo',
});
// 2. Generate AI response
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [
{ role: 'user', content: transcription.text }
],
});
// 3. Convert response to speech
const audio = await client.textToSpeech({
model: 'kokoro-82m',
input: response.choices[0].message.content,
voice: 'af_bella',
});
return audio;
}
```
## Supported Audio Formats
For transcription and translation:
* MP3
* WAV
* M4A
* OGG
* FLAC
* WebM
# Modular Pipeline Agent
Source: https://polargrid.mintlify.app/guides/voice-agent
Realtime voice agent that chains STT, LLM, and TTS with a streaming event channel (Beta)
# Modular Pipeline Agent
Beta. The connect endpoint and auth flow are not yet public. Contact PolarGrid for access.
## Overview
The Modular Pipeline Agent orchestrates three models from the PolarGrid catalog on each conversation turn: speech-to-text, an LLM, and text-to-speech. Audio streams in both directions. A parallel event channel carries JSON messages (transcripts, per-token LLM output, latency markers, errors). Any model returned by `GET /v1/models` with the matching modality can be used.
## Default models
* STT: `whisper-large-v3-turbo`
* LLM: `qwen-3.8-27b`
* TTS: `kokoro-82m` with voice `bm_george`
Overridable per session via request parameters (documented when the endpoint is public).
## Pipeline behavior
The agent runs on-device voice activity detection (VAD) to detect end-of-speech, transcribes the utterance, then streams LLM generation into TTS synthesis: TTS begins at the first sentence boundary of LLM output rather than after the full response. If the user starts speaking again while the agent is mid-response, the in-flight LLM and TTS are cancelled.
## Event reference
The server emits the following JSON events on the event channel. Some event types are emitted more than once per turn with different payloads — for example, `tts_complete` fires once when the first audio plays and again when synthesis finishes.
| Event | When | Payload fields |
| ------------------------ | ------------------------------ | ----------------------------------------------------------------------------- |
| `config` | On connect | `models: { stt, llm, tts }`, `vad: { threshold, silence_ms }` |
| `speech_end_detected` | VAD detects end of user speech | `server_timestamp_ms` |
| `transcript` | STT returns | `text`, `latency_ms`, `turn_id` |
| `llm_start`, `tts_start` | Pipeline markers | `turn_id` |
| `llm_token` | Each streamed LLM token | `token`, `index`, `ttft_ms` (first token only) |
| `llm_complete` | LLM generation finishes | `full_response`, `tokens`, `latency_ms`, `turn_id` |
| `tts_complete` (first) | First audio plays (TTFA) | `latency_ms` (TTFA), `total_pipeline_ms`, `turn_id`, `server_timestamp_ms` |
| `tts_complete` (second) | TTS generation ends | `latency_ms` (TTFA value preserved), `duration_ms`, `turn_summary`, `turn_id` |
| `metrics_stats` | Periodic | Aggregated `stt_latency`, `llm_ttft`, `tts_ttfa`, `pipeline_latency` |
| `error` | On error | `message` |
## Function calling
Function calling (tool use) is supported at the **chat completions API level** — the gateway's `/v1/chat/completions` endpoint supports `tools` and `tool_choice` parameters with models that have native tool-use support (`qwen-3.8-27b`).
The voice agent pipeline does **not** yet integrate function calling. Tool use in the voice agent is planned but not implemented — the pipeline currently streams text tokens only, with no tool-call detection, TTS pausing, or tool-result injection.
PersonaPlex does not support function calling (single audio-native model, no LLM step).
## Choosing between this and PersonaPlex
PersonaPlex uses a single multi-modal model and exposes a `persona` prompt and fixed voice IDs. The Modular Pipeline Agent runs three models you pick independently and exposes a per-turn event stream with latency markers. See the comparison table on the [PersonaPlex](/guides/personaplex) page.
## Access
Contact PolarGrid for Beta access.
# Voice Pipeline Quickstart
Source: https://polargrid.mintlify.app/guides/voice-pipeline-quickstart
Build a complete STT → LLM → TTS voice pipeline on PolarGrid
# Voice Pipeline Quickstart
This guide shows how to chain PolarGrid's three voice endpoints into a complete pipeline: transcribe speech, generate a response, and synthesize it back to audio — all on the same edge network.
All three models (Whisper, Qwen 3.5, Kokoro) run on the same edge node. No cross-provider latency, one auth token, one bill.
## Prerequisites
* A PolarGrid API key ([get one here](https://app.polargrid.ai/dashboard/settings?tab=api-keys))
* An audio file to transcribe (WAV, MP3, FLAC, M4A, OGG, or WebM)
* Node.js 18+ or Python 3.10+
## The Pipeline
```
Audio In → STT (Whisper) → LLM (Qwen 3.5) → TTS (Kokoro) → Audio Out
```
## JavaScript
```javascript theme={null}
import { PolarGrid } from '@polargrid/polargrid-sdk';
import { readFile, writeFile } from 'fs/promises';
const client = await PolarGrid.create({
apiKey: process.env.POLARGRID_API_KEY,
});
console.log(`Connected to: ${client.getRegionName()}`);
// Step 1: Transcribe audio → text
const audioInput = await readFile('input.wav');
const transcription = await client.transcribe({
file: new Blob([audioInput]),
model: 'whisper-large-v3-turbo',
});
console.log('User said:', transcription.text);
// Step 2: Generate a response with the LLM
const completion = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [
{
role: 'system',
content: 'You are a helpful voice assistant. Keep responses under 2 sentences.',
},
{ role: 'user', content: transcription.text },
],
});
const reply = completion.choices[0].message.content;
console.log('Assistant:', reply);
// Step 3: Synthesize the response to audio
const audioOutput = await client.textToSpeech({
model: 'kokoro-82m',
input: reply,
voice: 'af_bella',
responseFormat: 'pcm',
});
// Save as raw PCM (24 kHz, 16-bit, mono)
await writeFile('response.pcm', Buffer.from(audioOutput));
console.log('Audio saved to response.pcm');
// Convert to playable format:
// ffmpeg -f s16le -ar 24000 -ac 1 -i response.pcm response.wav
```
## Python
```python theme={null}
import asyncio
from polargrid import PolarGrid
async def voice_pipeline():
client = await PolarGrid.create(api_key="pg_your_api_key")
print(f"Connected to: {client.get_region_name()}")
# Step 1: Transcribe audio → text
with open("input.wav", "rb") as f:
transcription = await client.transcribe(
file=f,
request={"model": "whisper-large-v3-turbo"},
)
print(f"User said: {transcription.text}")
# Step 2: Generate a response with the LLM
completion = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [
{
"role": "system",
"content": "You are a helpful voice assistant. Keep responses under 2 sentences.",
},
{"role": "user", "content": transcription.text},
],
})
reply = completion.choices[0].message.content
print(f"Assistant: {reply}")
# Step 3: Synthesize the response to audio
audio_output = await client.text_to_speech({
"model": "kokoro-82m",
"input": reply,
"voice": "af_bella",
"response_format": "pcm",
})
with open("response.pcm", "wb") as f:
f.write(audio_output)
print("Audio saved to response.pcm")
# Convert to playable format:
# ffmpeg -f s16le -ar 24000 -ac 1 -i response.pcm response.wav
asyncio.run(voice_pipeline())
```
## cURL
```bash theme={null}
export API_KEY="pg_your_api_key"
EDGE="https://api.yto-01.edge.polargrid.ai"
# Step 1: Transcribe — single endpoint, query params select mode.
# Use ?sync=true for the lowest-latency blocking response in this voice loop.
TRANSCRIPT=$(curl -s -X POST \
"$EDGE/v1/audio/transcriptions?sync=true&model=whisper-large-v3-turbo" \
-H "Authorization: Bearer $API_KEY" \
-F file=@input.wav | jq -r .text)
echo "User said: $TRANSCRIPT"
# Step 2: LLM response
REPLY=$(curl -s -X POST "$EDGE/v1/chat/completions" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"model\": \"qwen-3.8-27b\",
\"messages\": [
{\"role\": \"system\", \"content\": \"You are a helpful voice assistant. Keep responses under 2 sentences.\"},
{\"role\": \"user\", \"content\": \"$TRANSCRIPT\"}
]
}" | jq -r '.choices[0].message.content')
echo "Assistant: $REPLY"
# Step 3: Synthesize to audio
curl -s -X POST "$EDGE/v1/audio/speech" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"model\": \"kokoro-82m\",
\"input\": \"$REPLY\",
\"voice\": \"af_bella\",
\"response_format\": \"pcm\"
}" -o response.pcm
echo "Audio saved to response.pcm"
# ffmpeg -f s16le -ar 24000 -ac 1 -i response.pcm response.wav
```
## Expected Latency
From a nearby region (e.g., Eastern North America → Toronto):
| Step | Typical Latency |
| ------------------- | ---------------- |
| STT (6s of audio) | \~500ms |
| LLM (TTFT) | \~120-250ms |
| TTS (2 sentences) | \~300-800ms |
| **Total perceived** | **\~900-1200ms** |
For real-time bidirectional voice (phone calls, live agents), see [PersonaPlex](/guides/personaplex) — it handles the full pipeline over a single WebSocket with streaming in both directions.
## Audio Format Notes
The examples above use `response_format: "pcm"` because real-time voice pipelines benefit from the lowest first-byte latency — PCM streams chunk-by-chunk, while `wav` and `mp3` are buffered until the full clip is synthesized.
If you'd rather get a playable file back directly, ask the server for a container:
```bash theme={null}
# Request WAV — RIFF/WAVE container, plays in any audio library
curl ... -d '{"...","response_format":"wav"}' -o response.wav
# Request MP3 — encoded server-side at 128 kbps CBR
curl ... -d '{"...","response_format":"mp3"}' -o response.mp3
```
`pcm`, `wav`, and `mp3` are the supported values. See [TTS API → Audio Format](/api-reference/text-to-speech#audio-format) for the full content-type / streaming table.
For HTTP-only stacks that can't pipe raw PCM chunks, set `stream: true` and `response_format: 'opus'` — same TTFB, smaller bytes on the wire. See the [TTS API streaming section](/api-reference/text-to-speech#streaming) for the full contract.
## Next Steps
Detailed TTS and STT endpoint reference
Real-time bidirectional voice agent over WebSocket
Stream LLM tokens as they generate
Available models and specifications
# Introduction
Source: https://polargrid.mintlify.app/introduction
GPU-powered AI inference at the edge with sub-30ms network latency
# Welcome to PolarGrid
PolarGrid is edge AI infrastructure that brings GPU-powered inference closer to your users. Run LLMs and voice AI (text-to-speech, speech-to-text, end-to-end voice agents) with ultra-low latency across our edge network.
Get your first API call working in 5 minutes
OpenAI wire-compatible endpoints — our open-source model catalog
npm install @polargrid/polargrid-sdk
pip install polargrid-sdk
## Why PolarGrid?
### Edge-First Architecture
Your inference requests are routed to the nearest GPU-equipped edge node, minimizing round-trip latency. Critical for real-time voice AI and interactive applications.
### OpenAI-Compatible API
Wire-compatible with OpenAI's API — same endpoints, request/response shapes, and `Bearer` auth, so apps migrate with a base-URL change. PolarGrid serves its own catalog of edge-deployed open-source models (see [Models](/models)), not a proxy to OpenAI, Gemini, or Claude — requests for cloud model IDs aren't served.
### Real-Time Voice
Sub-30ms network hop to the nearest edge node, enabling natural conversational AI experiences. Network latency is the round-trip time between your client and the edge -- inference latency (model processing time) is additional and varies by model and input size. See [Models](/models#performance) for performance details.
### Managed Model Infrastructure
PolarGrid handles model deployment and scaling across edge regions. Popular open-weight models are pre-loaded and ready to use — no provisioning or GPU management required.
## Available Regions
| Region | Location | ID |
| ------------- | -------------- | -------- |
| Toronto | Canada Central | `yto-01` |
| Montreal | Canada East | `yul-01` |
| Vancouver | Canada West | `yvr-02` |
| New York | US East | `nyc-01` |
| New York 02 | US East | `nyc-02` |
| Dallas | US Central | `dfw-01` |
| Dallas 02 | US Central | `dfw-02` |
| San Francisco | US West | `sfo-01` |
| Los Angeles | US West | `lax-01` |
| Seattle | US West | `sea-01` |
| Chicago | US Central | `chi-01` |
| Phoenix | US West | `phx-01` |
| Washington DC | US East | `was-01` |
| Miami | US East | `mia-01` |
| San Francisco | US West | `sfo-03` |
See [Regions](/guides/regions) for endpoint URLs, aliases, and auto-routing.
## Getting Help
* **Console**: app.polargrid.ai
* **Support**: [support@polargrid.ai](mailto:support@polargrid.ai)
* **GitHub**: github.com/polargrid-ai
# Models
Source: https://polargrid.mintlify.app/models
Available models, capabilities, and specifications
# Models
PolarGrid serves open-weight models on GPU-accelerated edge infrastructure. All models are available via our [OpenAI-compatible API](/api-reference/overview).
PolarGrid runs open-source models optimized for low-latency edge inference. These models are selected for speed and efficiency in real-time applications like voice AI. For workloads that require large cloud-hosted reasoning models (e.g., GPT-4, Gemini, Claude), use those providers directly — PolarGrid does not proxy requests to third-party APIs.
## LLM Models
### Qwen 3.8 27B
| | |
| ---------------- | ------------------------------------------------- |
| **Parameters** | 27B |
| **Quantization** | FP8 (native) |
| **Max context** | 262,144 tokens (256K) |
| **License** | Apache 2.0 |
| **Pricing** | $0.20 / 1M input tokens, $0.75 / 1M output tokens |
General-purpose large language model with strong performance across reasoning, coding, and multilingual tasks. Our largest deployed LLM, suitable for complex workloads where reply quality is the priority. Available fleet-wide on every edge node.
**Endpoint:** `POST /v1/chat/completions` with `"model": "qwen-3.8-27b"`
***
## Speech-to-Text Models
### Whisper Large V3 Turbo
| | |
| -------------- | ------------- |
| **Parameters** | 809M |
| **License** | Apache 2.0 |
| **Pricing** | \$0.004 / min |
OpenAI's Whisper model optimized for speed. Supports multilingual transcription with high accuracy.
**Endpoint:** `POST /v1/audio/transcriptions` with `"model": "whisper-large-v3-turbo"`
[Full model card →](/models/whisper-large-v3-turbo)
***
### Cohere Transcribe
| | |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| **Parameters** | 2B |
| **License** | [Apache 2.0](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026) |
| **Supported languages** | English, French, German, Italian, Spanish, Portuguese, Greek, Dutch, Polish, Chinese, Japanese, Korean, Vietnamese, Arabic |
| **Pricing** | \$0.004 / min |
High-accuracy multilingual transcription with support for 14 languages. Supports punctuation toggling. Available on `yvr-02` (Blackwell production).
**Endpoint:** `POST /v1/audio/transcriptions?sync=true` with `"model": "cohere-transcribe-03-2026"` (use the full model ID; the short alias `cohere-transcribe` is not routable)
**Latency** (yvr-02 Blackwell, external bench from a Vancouver-area laptop, 100 sync runs, 2026-05-27): server inference p50 **238 ms** / p95 **305 ms**, server-only RTF p50 **0.041** (24× faster than real time). End-to-end TTFB p50 **1068 ms** is upload-dominated — each request ships a \~300 KB WAV body before inference can start; the network leg is the upload time, not POP-to-client RTT. See the [Cohere Transcribe model page](/models/cohere-transcribe-03-2026) for the full breakdown.
***
## Text-to-Speech Models
### Hume AI TADA
| | |
| ----------------------- | -------------------------------------------------------------------------------------------------------------- |
| **Parameters** | \~4B (Llama 3.2 3B text base + audio components) |
| **License** | [Llama 3.2 Community License](https://huggingface.co/HumeAI/tada-3b-ml) |
| **Output** | 24 kHz mono — caller picks the container via `response_format` (`pcm`, `wav`, or `mp3`) |
| **Supported languages** | English, French, German, Spanish, Italian, Portuguese, Polish, Japanese, Arabic, Chinese |
| **Capabilities** | Cross-lingual voice cloning, speed control (batch only) |
| **Streaming** | ✓ chunked HTTP, `pcm` + `opus` (per-token via decoupled Triton handler; `speed` not honored in streaming mode) |
| **Pricing** | \$0.009 / min |
Expressive text-to-speech with cross-lingual voice cloning. Generates natural-sounding speech across 10 languages from a short reference clip.
**Endpoint:** `POST /v1/audio/speech` with `"model": "tada-3b-ml"`
***
### Kokoro 82M
| | |
| -------------- | ------------------------------ |
| **Parameters** | 82M |
| **License** | Apache 2.0 |
| **Streaming** | ✓ chunked HTTP, `pcm` + `opus` |
| **Pricing** | \$0.006 / min |
Lightweight, fast text-to-speech model. Ideal for low-latency voice applications where speed is critical.
**Endpoint:** `POST /v1/audio/speech` with `"model": "kokoro-82m"`
[Full model card →](/models/kokoro-82m)
***
## Voice Pipeline
### PersonaPlex
| | |
| -------------- | ---------------------------- |
| **Parameters** | 7B |
| **Pipeline** | STT + LLM + TTS (end-to-end) |
| **Pricing** | \$0.070 / min |
Integrated voice-to-voice pipeline that combines speech recognition, language model reasoning, and speech synthesis into a single low-latency stream. Billed by wall-clock duration.
See the [PersonaPlex guide](/guides/personaplex) for setup details.
***
## Performance
Latency benchmarks per model and region are actively being measured. Performance depends on:
* **Client proximity** to the nearest [edge region](/guides/regions)
* **Model size** — smaller models have lower TTFT and higher throughput
* **Request complexity** — token count, audio length, streaming vs. batch
The [autorouter](https://autorouter.polargrid.ai) optimizes for the lowest-latency region automatically. For detailed performance data, [contact us](mailto:hello@polargrid.ai).
## Custom Models
Enterprise customers can deploy custom fine-tuned models on PolarGrid infrastructure. PolarGrid handles provisioning and loading — contact us to discuss your model requirements.
For custom model deployments, contact [hello@polargrid.ai](mailto:hello@polargrid.ai).
See full pricing details and volume discounts
Full endpoint documentation
# Cohere Transcribe (03-2026)
Source: https://polargrid.mintlify.app/models/cohere-transcribe-03-2026
Multilingual high-accuracy speech-to-text on PolarGrid edge nodes
Cohere Transcribe (`cohere-transcribe-03-2026`) is a 2-billion-parameter multilingual speech-to-text model served on PolarGrid edge nodes via Triton's `python` backend. The voice pod runs `transformers >= 5.4` to support this model — see [`backend/edge-production-setup/CLAUDE.md`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/CLAUDE.md) for the pod split rationale.
* **HF repo:** [`CohereLabs/cohere-transcribe-03-2026`](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026)
* **Modality:** Speech-to-Text (streaming + sync)
* **Backend:** Triton `python` (voice pod)
* **Model ID:** `cohere-transcribe-03-2026` (use the full ID; the short alias `cohere-transcribe` is not routable on the gateway)
- **Available regions:** all regions except `dfw-02` — see [Model availability](/guides/model-availability)
## Headline benchmark
`POST /v1/audio/transcriptions?stream=true` is the live surface. The server emits a `text/event-stream` of `transcript.text.delta` events as the cohere handler's internal decode loop completes each window, then closes with a single `transcript.text.done` event. Consumers can render the rolling transcript immediately instead of waiting for the final result.
| Measurement | p50 | p95 |
| --------------------------------------------------- | --------- | ------- |
| **TTFT (response headers → first non-empty delta)** | **44 ms** | 62 ms |
| Time to `done` event (response headers → done) | 560 ms | 856 ms |
| Interim work (first delta → done) | 517 ms | 816 ms |
| Delta events per request | 4 | 7 (max) |
| Partial deltas (text != final) | 4 | — |
| e2e total (POST → `[DONE]`) | 2510 ms | 3523 ms |
| **RTF (e2e ÷ audio duration)** | **0.42** | 0.53 |
| well\_formed | 100 / 100 | — |
*Bench: 100 streaming transcription runs against `https://api.yvr-02.edge.polargrid.ai`, captured 2026-05-28 from a Vancouver-area laptop. Inputs were 5 short utterances (4.2 – 7.7 s, 24 kHz mono WAV, \~200–350 KB each) pre-synthesized via `tada-3b-ml` on the same node — see [`bench/cohere-transcribe-03-2026/synthesize_inputs.py`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/cohere-transcribe-03-2026). Raw runs: [`benchmarks/yvr-02-2026-05-28/cohere-transcribe-03-2026/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-05-28/cohere-transcribe-03-2026/cohere-transcribe-03-2026_streaming_bench.json).*
> **Delta cadence is chunk-driven.** 4.2 – 5.9 s clips emit 4 deltas, 7.2 – 7.7 s clips emit 7. Each delta carries a non-overlapping span of the transcript; concatenating them reconstructs the final string. The first delta arriving at 44 ms after response headers is the meaningful TTFT for live-captioning pipelines. The gateway does not emit `X-Pg-Inference-Ms` on the stream surface yet, so server-only inference cannot be quoted client-side for streaming today.
## How this compares
| Provider | Streaming first-partial p50 | Sync RTF | Notes | Source |
| ------------------------------------------------------ | -------------------------------------------- | --------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------- |
| **PolarGrid `cohere-transcribe-03-2026` on Blackwell** | **44 ms** after response headers | **0.041** server-only | 14 languages, sync + streaming surfaces | this card |
| Deepgram Nova-3 | sub-300 ms over WebSocket from end-of-speech | n/a | WebSocket protocol streams audio in continuously | [artificialanalysis.ai](https://artificialanalysis.ai/speech-to-text/models/deepgram) |
| AssemblyAI Universal-3 | n/a | \~0.008 to 0.05 batch | Batch transcription tier | [assemblyai.com](https://www.assemblyai.com/benchmarks) |
PolarGrid's 44 ms streaming TTFT is gated by the multipart audio upload arriving first; total client wall-clock to first partial is closer to 1970 ms p50 (TTFB 1926 ms + first-delta 44 ms) on a 300 KB WAV. Deepgram Nova-3 measures streaming TTFT from end-of-speech because the WebSocket protocol streams audio in continuously, which removes the upload component entirely. For mic-to-screen pipelines that need sub-300 ms first-partial from end-of-speech, an audio-streaming-in protocol (WebSocket or chunked-upload) is the missing piece on PolarGrid today. For batch-upload workloads PolarGrid's server-only RTF of 0.041 is in the same tier as AssemblyAI Universal-3 batch.
## Quickstart
Edge endpoints accept your raw `pg_*` API key as a bearer token — no token exchange. See [Authentication](/authentication).
```bash cURL theme={null}
curl -X POST "https://api.yvr-02.edge.polargrid.ai/v1/audio/transcriptions?sync=true&model=cohere-transcribe-03-2026" \
-H "Authorization: Bearer $POLARGRID_API_KEY" \
-F "file=@input.wav"
```
```typescript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
import fs from "node:fs";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
const result = await client.audioTranscriptions({
model: "cohere-transcribe-03-2026",
file: fs.createReadStream("input.wav"),
// Default mode is async (returns job_id). Pass sync to block on the result.
sync: true,
});
console.log(result.text);
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
with open("input.wav", "rb") as f:
result = await client.audio_transcriptions({
"model": "cohere-transcribe-03-2026",
"file": f,
"sync": True,
})
print(result["text"])
```
## Endpoint modes
`POST /v1/audio/transcriptions` has three modes selected by query params (not multipart fields):
| Mode | Query | Response | Use when |
| ------------------- | -------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Streaming** | `?stream=true` | `text/event-stream` of delta + done events | Live captioning; sub-utterance partial transcripts as the model decodes. **This is what the headline bench above measures.** |
| **Sync** | `?sync=true` | `200` with the formatted transcript directly | Voice-agent path that needs the answer in one round trip. |
| **Async (default)** | *none* | `202` with `{job_id, poll_url}`; poll via `GET ?job_id=...` | Background batch transcription; no caller blocking. |
`stream` and `sync` are mutually exclusive — passing both returns `400`. Streaming requires `response_format` in `{json, text}`.
### Sync benchmark (`?sync=true`)
The blocking surface holds the connection open until inference completes, then ships the whole JSON response. Client-side TTFB ≈ total wall-clock, and the meaningful split is **server inference time** vs **network leg** (which for STT is dominated by the audio upload). Raw runs: [`benchmarks/yvr-02-2026-05-27/cohere-transcribe-03-2026/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-05-27/cohere-transcribe-03-2026/cohere-transcribe-03-2026_bench.json).
| Measurement | p50 | p95 |
| --------------------------------------- | ---------- | ------- |
| End-to-end TTFB (with network + upload) | 1068 ms | 1535 ms |
| **Server-only inference** | **238 ms** | 305 ms |
| *Network leg (e2e − server)* | *831 ms* | — |
| Body transfer (JSON response) | 0.7 ms | 1.6 ms |
| RTF (server inference ÷ audio duration) | 0.041 | 0.056 |
Server-only timing comes from the `X-Pg-Inference-Ms` response header (PR #507), available on the sync surface.
> **The network leg is upload-dominated.** Each request ships a multi-second WAV file before inference can begin, so the 831 ms p50 network figure is largely the upload time of a \~300 KB body — not POP-to-client RTT. For shorter clips (sub-2 s) the network leg shrinks proportionally. Quote the **server-only RTF** when comparing inference throughput against centralized providers.
## Capabilities
| Field | Value | |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------- | ------- |
| Endpoint | `POST /v1/audio/transcriptions` | |
| Multipart field | `file` (required) | |
| Query params | `model`, `language`, `prompt`, `temperature`, `response_format`, `punctuation`, `stream`, `sync` | |
| `response_format` | `json` (default), `text`, `srt`, `vtt`, `verbose_json` | |
| Languages | English, French, German, Italian, Spanish, Portuguese, Greek, Dutch, Polish, Chinese, Japanese, Korean, Vietnamese, Arabic | |
| Punctuation toggle | Yes — pass \`punctuation=true | false\` |
| Max batch size | 1 | |
| Backend pod | `inference-backend-triton-voice` | |
## Response timing headers
PR #507 added two response headers that bench harnesses and observability tooling can read to get a server-only inference time without inferring it from the body:
| Header | Value |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `X-Pg-Inference-Ms` | Integer milliseconds — wall-clock around the `transcribe()` call inside the gateway's `_handle_sync` (i.e., excludes the audio upload + response transfer). |
| `Server-Timing` | Standard-shape `inference;dur=` entry carrying the same number. |
These let callers compute the network leg as `(client wall-clock) − (X-Pg-Inference-Ms)`, the same e2e-vs-server split available for LLM via `pg_metadata`.
## Model identifier
Call this model with the full id `cohere-transcribe-03-2026` at `/v1/audio/transcriptions`. The short alias `cohere-transcribe` is **not routable** on the gateway (returns 404). The HuggingFace repo id `CohereLabs/cohere-transcribe-03-2026` is accepted at `/v1/models/load` for hot-loading purposes but does not resolve at inference time.
## Notes
* License: [Apache 2.0](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026/blob/main/LICENSE) (no auth required to pull weights).
* Voice pod isolation: this model needs `transformers >= 5.4`, which conflicts with hume-tada's `< 5` pin and the LLM pod's vLLM 0.17.x. That's why PolarGrid splits voice/LLM/TADA into three Triton pods — see [`backend/edge-production-setup/CLAUDE.md`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/CLAUDE.md).
* For lower full-utterance latency, [`whisper-large-v3-turbo`](/models/whisper-large-v3-turbo) is the alternative — Cohere's edge is multilingual coverage + accuracy on accented speech.
## See also
* [Speech-to-Text API](/api-reference/speech-to-text) — endpoint reference, formats, streaming contract
* [Voice AI guide](/guides/voice) — building voice agents on PolarGrid
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models
# Kokoro 82M
Source: https://polargrid.mintlify.app/models/kokoro-82m
Low-latency preset-voice streaming TTS on PolarGrid edge nodes
Kokoro 82M (`kokoro-82m`) is an 82M-parameter text-to-speech model served on PolarGrid edge nodes via Triton's `python` backend. It is a **preset-voice** model — pick one of a fixed catalog of named voices — which makes it the low-latency counterpart to [`tada-3b-ml`](/models/tada-3b-ml)'s voice-cloning. It is co-resident on the voice pod with [`whisper-large-v3-turbo`](/models/whisper-large-v3-turbo) and [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) — see [`backend/edge-production-setup/CLAUDE.md`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/CLAUDE.md) for the pod layout.
* **HF repo:** [`hexgrad/Kokoro-82M`](https://huggingface.co/hexgrad/Kokoro-82M)
* **Modality:** Text-to-Speech (streaming)
* **Backend:** Triton `python` (voice pod)
* **Parameters:** 82M
- **Available regions:** all regions except `dfw-02` — see [Model availability](/guides/model-availability)
## Headline benchmark
Kokoro exposes a chunked-HTTP streaming transport for `/v1/audio/speech`.
| Measurement | p50 | p95 |
| -------------------------------------------------- | ------------------- | ------- |
| **End-to-end TTFA (with network)** | **158 ms** | 196 ms |
| **Server-only TTFA (gateway → first triton byte)** | **46 ms** | 63 ms |
| *Network leg of TTFA (e2e − server)* | *109 ms* | 134 ms |
| Full-utterance latency (TTLB, client wall-clock) | 696 ms | 1025 ms |
| Real-time factor (RTF) | 0.100 | 0.149 |
| streaming\_verdict | streaming (100/100) | — |
*Streaming `/v1/audio/speech` (`stream: true`, `pcm`, voice `bm_george`), 100 runs against `https://api.yvr-02.edge.polargrid.ai`, captured 2026-06-02 from a Vancouver-area laptop over the public internet. TTFA is the time to the first audio byte arriving at the client — identical to TTFB since the response body is raw PCM. Server-only TTFA comes from the `X-Pg-First-Byte-Ms` response header ([PR #507](https://github.com/PolarGrid-AI/polargrid-monorepo/pull/507)), the gateway's measured time to the first PCM chunk returned by triton. Total synthesis time (TTLB) is not exposed via header and stays a client-side wall-clock number. RTF = synthesis wall-clock ÷ audio duration; below 1.0 is faster than real time. All 100/100 runs returned a `streaming` verdict (54–234 chunks per utterance). Raw runs: [`benchmarks/yvr-02-2026-06-02/kokoro-82m/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-02/kokoro-82m/kokoro-82m_bench.json). Harness: [`bench/kokoro-82m/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/kokoro-82m).*
## How this compares
Same node, same harness, same `/v1/audio/speech` streaming surface as the [`tada-3b-ml`](/models/tada-3b-ml) bench in `benchmarks/yvr-02-2026-05-27/`:
| Model | server TTFA p50 | e2e TTFA p50 | e2e TTLB p50 | RTF p50 |
| ---------------------- | --------------- | ------------ | ------------ | --------- |
| `tada-3b-ml` (3B) | 238 ms | 352 ms | 919 ms | 0.164 |
| **`kokoro-82m` (82M)** | **46 ms** | **158 ms** | **696 ms** | **0.100** |
Kokoro's 82M model synthesizes the first audio chunk \~5× faster server-side (46 vs 238 ms) and finishes the full utterance sooner at a lower real-time factor. The \~109 ms network leg is identical across both — same laptop, same POP. Pick kokoro for lowest-latency synthesis from a fixed voice catalog; pick tada when you need a specific cloned voice or a non-English language. For external-provider TTS comparisons (Cartesia, ElevenLabs, Hume), see the [`tada-3b-ml` card](/models/tada-3b-ml#how-this-compares).
## Quickstart
Edge endpoints accept your raw `pg_*` API key as a bearer token — no token exchange. See [Authentication](/authentication). Replace `` with your edge region, or discover the nearest one via the [autorouter](/api-reference/overview#picking-a-region).
```bash cURL theme={null}
curl -X POST https://api..edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer $POLARGRID_API_KEY" \
-H "Content-Type: application/json" \
--no-buffer \
-d '{
"model": "kokoro-82m",
"input": "Hello from PolarGrid.",
"voice": "bm_george",
"response_format": "pcm",
"stream": true
}' \
--output speech.pcm
```
```javascript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
for await (const chunk of client.textToSpeechStream({
model: "kokoro-82m",
input: "Hello from PolarGrid.",
voice: "bm_george",
responseFormat: "opus",
})) {
audioPlayer.appendChunk(chunk);
}
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
async for chunk in client.text_to_speech_stream({
"model": "kokoro-82m",
"input": "Hello from PolarGrid.",
"voice": "bm_george",
"response_format": "opus",
}):
audio_player.append_chunk(chunk)
```
## Capabilities
| Field | Value |
| --------------- | ---------------------------------------------------------------------------------------------------------- |
| Endpoint | `POST /v1/audio/speech` |
| Audio output | 24 kHz, 16-bit, mono |
| Streaming | Yes — chunked HTTP, `pcm` and `opus` (`stream: true`); audio delivered incrementally as synthesis proceeds |
| Batch formats | `pcm`, `wav`, `mp3` |
| Voice model | Fixed catalog of named preset voices (no cloning) |
| `speed` control | `0.25`–`4.0` multiplier |
| Max batch size | 1 |
`voice_transcript` and `language` are `tada-3b-ml`-only fields and are ignored by `kokoro-82m`.
## Voices — preset catalog
Kokoro uses **named preset voices**, not reference-clip cloning. The PolarGrid SDKs expose eight presets:
| Voice ID | Accent / gender |
| ------------- | ------------------------ |
| `af_bella` | American English, female |
| `af_sarah` | American English, female |
| `am_adam` | American English, male |
| `am_michael` | American English, male |
| `bf_emma` | British English, female |
| `bf_isabella` | British English, female |
| `bm_george` | British English, male |
| `bm_lewis` | British English, male |
`voice` is required; the PolarGrid SDKs and CLI default to `af_bella`. An invalid voice name returns `502 Bad Gateway` (synthesis failure), never an empty `200`. See the [Text-to-Speech API reference](/api-reference/text-to-speech#voices) and the [Voice AI guide](/guides/voice) for the full upstream voice list.
## Streaming
Pass `stream: true` for chunked audio over a single HTTP response. Streaming formats are `pcm` (default for raw HTTP callers) and `opus`; the PolarGrid SDKs default streaming requests to `opus`. Requesting `wav` or `mp3` with `stream: true` returns `400 Bad Request`.
Audio is delivered incrementally as kokoro's pipeline produces it — the bench above observed 54–234 PCM chunks per utterance with a `streaming` verdict on every run, so the first audio arrives well before synthesis finishes. See the [Text-to-Speech API reference](/api-reference/text-to-speech#streaming) for the full streaming contract — response headers, truncated-stream detection, and the per-format table.
## Model identifier
Call this model with the canonical id `kokoro-82m` at `/v1/audio/speech`. It has no short alias. The HuggingFace repo id `hexgrad/Kokoro-82M` is accepted at `/v1/models/load` for hot-loading but does not resolve at inference time.
## Notes
* License: [Apache 2.0](https://huggingface.co/hexgrad/Kokoro-82M).
* For a specific cloned voice or a non-English language, use [`tada-3b-ml`](/models/tada-3b-ml) — kokoro is the choice for lowest-latency synthesis from a fixed English voice catalog.
## See also
* [Text-to-Speech API](/api-reference/text-to-speech) — endpoint reference, formats, streaming contract
* [Voice AI guide](/guides/voice) — building voice agents on PolarGrid
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models
# Qwen3.5 27B
Source: https://polargrid.mintlify.app/models/qwen-3.5-27b
Retired — superseded by Qwen3.8 27B. Kept for reference.
**Retired.** `qwen-3.5-27b` is no longer served on any PolarGrid edge node — the
fleet completed its move to [`qwen-3.8-27b`](/models/qwen-3.8-27b) on
2026-08-20, and requests for this id now return `404 model_not_loaded`. Update
the `model` field in your requests to `qwen-3.8-27b`. This page is kept for
reference; the benchmark figures below describe the retired model.
Qwen3.5 27B (`qwen-3.5-27b`) is a 27-billion-parameter text LLM served on PolarGrid edge nodes via Triton's `vllm_backend`. Weights ship pre-quantized to FP8 (\~28 GB VRAM) and load directly on PolarGrid's Blackwell edge GPUs without runtime requantization.
* **HF repo:** [`Qwen/Qwen3.5-27B-FP8`](https://huggingface.co/Qwen/Qwen3.5-27B-FP8)
* **Modality:** Text LLM
* **Backend:** Triton `vllm` (LLM pod)
* **Available regions:** fleet-wide — see [Model availability](/guides/model-availability)
## Headline benchmark
We publish **two** numbers side by side. End-to-end is what your application actually experiences (request → response, network included). Server-only is what the GPU spends on inference (apples-to-apples vs centralized providers' published "inference-only" figures). The gap is the latency PolarGrid's `yvr-02` PoP eliminates by being at the edge.
| Measurement | TTFT p50 | TTFT p95 | Throughput p50 |
| --------------------------------- | ---------- | -------- | -------------- |
| **End-to-end (with network)** | **177 ms** | 243 ms | **29.0 tok/s** |
| **Server-only (no network)** | **71 ms** | 131 ms | — |
| *Network overhead (e2e − server)* | *106 ms* | — | — |
*Bench: 100 streaming chat-completion runs against `https://api.yvr-02.edge.polargrid.ai`, captured 2026-07-09 from a local macOS host over the public internet. End-to-end is client wall-clock; server-only is read from the gateway's `pg_metadata` SSE event (`inference_ttft_ms` / `inference_total_ms`). Reasoning mode off (default). CUDA graphs on (`enforce_eager=false`, the current fleet default). The yvr-02 node ships on RTX 6000 Pro Blackwell 96 GB — roughly 2.4× the tok/s and less than half the e2e total latency of the earlier yvr-01 (L40S-class) baseline at `benchmarks/qwen-2026-05-01/27b/llm_bench.json`. Raw runs: `benchmarks/yvr-02-2026-07-09/27b/llm_bench.json`.*
> **Apples-to-apples disclaimer.** Other providers usually publish only their server-side number; comparing it to our **server-only** row is the fair baseline. Our **end-to-end** row is what you'll see from a customer-side request because PolarGrid runs at the edge. The network row above shows exactly how much that's worth in milliseconds.
## How this compares
| Provider | TTFT p50 | Throughput p50 | Source |
| ----------------------------------------- | --------------------------------- | -------------- | ---------------------------------------------------------------------------------------------- |
| **PolarGrid `qwen-3.5-27b` on Blackwell** | **177 ms** e2e / **71 ms** server | **29.0 tok/s** | this card |
| Claude Sonnet 4.5 | \~1600 ms | 47.6 tok/s | [artificialanalysis.ai](https://artificialanalysis.ai/models/claude-4-5-sonnet) |
| gpt-4o | \~850 ms | 135 tok/s | [artificialanalysis.ai](https://artificialanalysis.ai/models/gpt-4o) |
| Cerebras (specialty silicon) | n/a | \~2100 tok/s | [cerebras.ai](https://www.cerebras.ai/blog/cerebras-inference-3x-faster) |
| Groq with speculative decoding | n/a | \~1665 tok/s | [artificialanalysis.ai](https://artificialanalysis.ai/models/llama-3-3-instruct-70b/providers) |
PolarGrid wins on TTFT end-to-end against frontier-reasoning providers because of edge proximity (106 ms p50 network leg vs centralized regions). PolarGrid is 4 to 70 times behind specialty silicon on raw throughput; RTX 6000 Pro Blackwell workstation FLOPS are below H100 and H200 datacenter FLOPS. CUDA graphs are enabled (`enforce_eager=false`, the current fleet default), which recovered about 25 ms of server TTFT versus the earlier eager-mode baseline at `benchmarks/qwen-2026-07-07/27b/llm_bench.json`.
## Quickstart
```bash cURL theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.5-27b",
"messages": [{"role": "user", "content": "Say hi in one short sentence."}],
"stream": true,
"max_tokens": 32
}'
```
```typescript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
for await (const chunk of client.chatCompletionStream({
model: "qwen-3.5-27b",
messages: [{ role: "user", content: "Say hi in one short sentence." }],
maxTokens: 32,
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
}
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
async for chunk in client.chat_completion_stream({
"model": "qwen-3.5-27b",
"messages": [{"role": "user", "content": "Say hi in one short sentence."}],
"max_tokens": 32,
}):
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
## Capabilities
| Field | Value |
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| Context window | 8192 tokens |
| Streaming | Yes (SSE via `stream: true`) |
| Function calling / tools | Yes (Hermes-style; see "Function calling" below) |
| Structured output (`response_format`) | Yes — `json_object` and `json_schema` (vLLM `structured_outputs` constrained decoding) |
| Logprobs | No (vllm\_backend exposes only `text_output` over Triton; not surfaced) |
| Sampling controls | `temperature`, `top_p`, `top_k`, `min_p`, `frequency_penalty`, `presence_penalty`, `repetition_penalty`, `seed`, `stop` |
| Reasoning ("thinking") mode | Off by default; opt in via `"enable_thinking": true` in the request body |
## Function calling
Pass OpenAI-shape `tools` and the model returns a `tool_calls` array on the assistant message (or as a `delta.tool_calls` chunk when streaming). The gateway speaks Qwen's Hermes tool-call template under the hood.
```bash cURL theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.5-27b",
"messages": [{"role": "user", "content": "Whats the weather in Tokyo?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"tool_choice": "auto"
}'
```
```typescript JavaScript theme={null}
const reply = await client.chatCompletion({
model: "qwen-3.5-27b",
messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
tools: [{
type: "function",
function: {
name: "get_weather",
description: "Get the current weather in a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"]
}
}
}],
tool_choice: "auto",
});
const call = reply.choices[0].message.tool_calls?.[0];
// call.function.name === "get_weather"
// JSON.parse(call.function.arguments) === { city: "Tokyo" }
```
`tool_choice` accepts `"auto"` (model decides), `"none"` (force plain text), `"required"` (force a tool call), or `{ "type": "function", "function": { "name": "" } }` to force a specific tool.
After invoking the tool yourself, append a `role: "tool"` message containing the result and re-call the model:
```json theme={null}
{
"model": "qwen-3.5-27b",
"messages": [
{"role": "user", "content": "Weather in Tokyo?"},
{"role": "assistant", "tool_calls": [
{"id": "call_abc", "type": "function",
"function": {"name": "get_weather", "arguments": "{\"city\":\"Tokyo\"}"}}
]},
{"role": "tool", "tool_call_id": "call_abc",
"content": "{\"temp_c\": 22, \"sky\": \"sunny\"}"}
]
}
```
## Structured output (JSON mode)
Use `response_format` to force the model to emit valid JSON. Backed server-side by vLLM's `structured_outputs` constrained decoding, so the output is guaranteed to parse.
```bash theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.5-27b",
"messages": [{"role": "user", "content": "Give me a JSON object describing a cat."}],
"response_format": {
"type": "json_schema",
"json_schema": {
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"age_years": {"type": "integer"},
"color": {"type": "string"}
},
"required": ["name", "age_years", "color"]
}
}
}
}'
```
`{"type": "json_object"}` accepts any valid JSON; `json_schema` constrains it to your schema.
### Reasoning mode
Qwen3.5 ships with a "thinking" mode that emits a `...` reasoning trace before the user-visible answer. PolarGrid's `/v1/chat/completions` endpoint disables this by default to keep first-token latency low. The 27B variant runs the same toggle, with deeper reasoning quality at the cost of longer generation time.
To enable thinking on a per-request basis:
```json theme={null}
{
"model": "qwen-3.5-27b",
"messages": [{"role": "user", "content": "..."}],
"enable_thinking": true
}
```
## Model identifier
Call this model with the canonical id `qwen-3.5-27b` at all inference endpoints (`/v1/chat/completions`, `/v1/completions`). The HuggingFace repo id `Qwen/Qwen3.5-27B-FP8` is accepted at `/v1/models/load` for hot-loading purposes but does **not** resolve at inference time — use the canonical id for chat and completions calls.
## Notes
* License: [Apache 2.0](https://huggingface.co/Qwen/Qwen3.5-27B-FP8/blob/main/LICENSE) (no auth required to pull weights).
* Native FP8 — no runtime quantization step at load.
* VRAM is tight: a single 46 GB L40S can host this model **or** the voice stack, not both. Multi-GPU edges pin 27B to its own GPU; see `backend/edge-production-setup/CLAUDE.md` for the layout matrix.
## See also
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models
* [`/v1/chat/completions`](/api-reference/chat-completions) — endpoint reference
# Qwen3.6 35B-A3B
Source: https://polargrid.mintlify.app/models/qwen-3.6-35b-a3b
Qwen3.6 35B-A3B MoE text LLM on vLLM with native FP8 weights
Qwen3.6 35B-A3B (`qwen-3.6-35b-a3b`) is a Mixture-of-Experts text LLM served on PolarGrid edge nodes via Triton's `vllm_backend`. It has **35 billion total parameters** but activates only **\~3 billion per token** (256 experts, 8 routed + 1 shared active). Weights ship pre-quantized to FP8 (\~35 GB VRAM) and load directly on PolarGrid's Blackwell edge GPUs without runtime requantization.
* **HF repo:** [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8)
* **Modality:** Text LLM (MoE)
* **Backend:** Triton `vllm` (LLM pod)
- **Available regions:** limited availability (customer pilot) — [contact us](https://polargrid.ai/contact) for access
## Headline benchmark
We publish **two** numbers side by side. End-to-end is what your application actually experiences (request → response, network included). Server-only is what the GPU spends on inference (apples-to-apples vs centralized providers' published "inference-only" figures). The gap is the latency PolarGrid's edge PoP eliminates by being close to the caller.
| Measurement | TTFT p50 | TTFT p95 | Throughput p50 |
| --------------------------------- | ---------- | -------- | -------------- |
| **End-to-end (with network)** | **212 ms** | 238 ms | **15.8 tok/s** |
| **Server-only (no network)** | **163 ms** | 174 ms | — |
| *Network overhead (e2e − server)* | *49 ms* | — | — |
*Bench: 100 streaming chat-completion runs against the Montreal customer-pilot node (since retired from the public fleet), captured 2026-06-09 from yvr-01 (Vancouver) over the public internet. End-to-end is client wall-clock; server-only is read from the gateway's `pg_metadata` SSE event (`inference_ttft_ms` / `inference_total_ms`). Raw runs: [`benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json).*
> **Apples-to-apples disclaimer.** Other providers usually publish only their server-side number; comparing it to our **server-only** row is the fair baseline. Our **end-to-end** row is what you'll see from a customer-side request because PolarGrid runs at the edge — the network row above shows exactly how much that's worth in milliseconds.
## Quickstart
```bash cURL theme={null}
curl https://api..edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.6-35b-a3b",
"messages": [{"role": "user", "content": "Say hi in one short sentence."}],
"stream": true,
"max_tokens": 32
}'
```
```typescript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
for await (const chunk of client.chatCompletionStream({
model: "qwen-3.6-35b-a3b",
messages: [{ role: "user", content: "Say hi in one short sentence." }],
maxTokens: 32,
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
}
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
async for chunk in client.chat_completion_stream({
"model": "qwen-3.6-35b-a3b",
"messages": [{"role": "user", "content": "Say hi in one short sentence."}],
"max_tokens": 32,
}):
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
## Capabilities
| Field | Value |
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| Architecture | Mixture-of-Experts (256 experts, 8 routed + 1 shared active; \~3B active / 35B total) |
| Context window | 8192 tokens (served; native context is larger, capped here to bound KV-cache VRAM) |
| Streaming | Yes (SSE via `stream: true`) |
| Function calling / tools | Yes (Hermes-style; see "Function calling" below) |
| Structured output (`response_format`) | Yes — `json_object` and `json_schema` (vLLM constrained decoding) |
| Logprobs | No (vllm\_backend exposes only `text_output` over Triton; not surfaced) |
| Sampling controls | `temperature`, `top_p`, `top_k`, `min_p`, `frequency_penalty`, `presence_penalty`, `repetition_penalty`, `seed`, `stop` |
| Reasoning ("thinking") mode | Off by default; opt in via `"enable_thinking": true` in the request body |
## Function calling
Pass OpenAI-shape `tools` and the model returns a `tool_calls` array on the assistant message (or as a `delta.tool_calls` chunk when streaming). The gateway speaks Qwen's Hermes tool-call template under the hood.
```bash cURL theme={null}
curl https://api..edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.6-35b-a3b",
"messages": [{"role": "user", "content": "Whats the weather in Tokyo?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"tool_choice": "auto"
}'
```
```typescript JavaScript theme={null}
const reply = await client.chatCompletion({
model: "qwen-3.6-35b-a3b",
messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
tools: [{
type: "function",
function: {
name: "get_weather",
description: "Get the current weather in a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"]
}
}
}],
tool_choice: "auto",
});
const call = reply.choices[0].message.tool_calls?.[0];
// call.function.name === "get_weather"
// JSON.parse(call.function.arguments) === { city: "Tokyo" }
```
`tool_choice` accepts `"auto"` (model decides), `"none"` (force plain text), `"required"` (force a tool call), or `{ "type": "function", "function": { "name": "" } }` to force a specific tool.
## Structured output (JSON mode)
Use `response_format` to force the model to emit valid JSON. Backed server-side by vLLM constrained decoding, so the output is guaranteed to parse.
```bash theme={null}
curl https://api..edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.6-35b-a3b",
"messages": [{"role": "user", "content": "Give me a JSON object describing a cat."}],
"response_format": {
"type": "json_schema",
"json_schema": {
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"age_years": {"type": "integer"},
"color": {"type": "string"}
},
"required": ["name", "age_years", "color"]
}
}
}
}'
```
`{"type": "json_object"}` accepts any valid JSON; `json_schema` constrains it to your schema.
### Reasoning mode
Qwen3.6 ships with a "thinking" mode that emits a `...` reasoning trace before the user-visible answer. PolarGrid's `/v1/chat/completions` endpoint disables this by default to keep first-token latency low. Opt in per request:
```json theme={null}
{
"model": "qwen-3.6-35b-a3b",
"messages": [{"role": "user", "content": "..."}],
"enable_thinking": true
}
```
## Model identifier
Call this model with the canonical id `qwen-3.6-35b-a3b` at all inference endpoints (`/v1/chat/completions`, `/v1/completions`). The HuggingFace repo id `Qwen/Qwen3.6-35B-A3B-FP8` and the short alias `qwen-3.6-a3b` are accepted at `/v1/models/load` for hot-loading, but inference calls should use the canonical id.
## Aliases
The following caller-facing aliases resolve to `qwen-3.6-35b-a3b`:
| Alias | Resolves to |
| -------------------------- | ------------------ |
| `qwen-3.6-a3b` | `qwen-3.6-35b-a3b` |
| `Qwen/Qwen3.6-35B-A3B-FP8` | `qwen-3.6-35b-a3b` |
## Notes
* **MoE efficiency:** only \~3B of the 35B parameters activate per token, so throughput is closer to a small dense model while quality tracks the full 35B. All expert weights remain resident in VRAM (\~35 GB at FP8).
* **Native FP8** — no runtime quantization step at load.
* **Vision (image input)** is served on the `sfo-02` staging node — OpenAI `image_url` content parts, with context reduced during bring-up. Customer-pilot deployments run the **text-only** path (the vision tower is not loaded). See [Chat Completions → Vision](/api-reference/chat-completions#vision-image-input). `enforce_eager=true` works around a vLLM CUDA-graph path on this model's Gated-DeltaNet hybrid attention, same workaround the retired `qwen-3.5-27b` needed. The current fleet LLM, [`qwen-3.8-27b`](/models/qwen-3.8-27b), does **not** need it — CUDA-graph capture was re-validated clean on its hybrid attention and it runs `enforce_eager=false`.
* Offered as a customer pilot; co-located with on-edge embeddings for retrieval workloads.
## See also
* [Qwen3.8 27B](/models/qwen-3.8-27b) — sibling dense LLM
* [Authentication](/authentication) — how to mint a JWT from your API key
* [`/v1/models`](/api-reference/models) — list all available models
* Bench source: [`backend/edge-production-setup/bench/qwen-3.6-35b-a3b/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/qwen-3.6-35b-a3b)
# Qwen3.8 27B
Source: https://polargrid.mintlify.app/models/qwen-3.8-27b
Qwen3.8 27B text LLM on vLLM with native FP8 weights
Qwen3.8 27B (`qwen-3.8-27b`) is a 27-billion-parameter text LLM served on PolarGrid edge nodes via Triton's `vllm_backend`. Weights ship pre-quantized to FP8 (\~28 GB VRAM) and load directly on PolarGrid's Blackwell edge GPUs without runtime requantization.
* **HF repo:** [`Qwen/Qwen3.8-27B-FP8`](https://huggingface.co/Qwen/Qwen3.8-27B-FP8)
* **Modality:** Text LLM
* **Backend:** Triton `vllm` (LLM pod)
* **Available regions:** fleet-wide — see [Model availability](/guides/model-availability)
## Headline benchmark
We publish **two** numbers side by side. End-to-end is what your application actually experiences (request → response, network included). Server-only is what the GPU spends on inference (apples-to-apples vs centralized providers' published "inference-only" figures). The gap is the latency PolarGrid's `yvr-02` PoP eliminates by being at the edge.
| Measurement | TTFT p50 | TTFT p95 | Throughput p50 |
| --------------------------------- | --------- | -------- | -------------- |
| **Server-only (no network)** | **72 ms** | 141 ms | **29.5 tok/s** |
| **End-to-end (with network)** | 284 ms | 363 ms | — |
| *Network overhead (e2e − server)* | *210 ms* | — | — |
*Bench: 60 streaming chat-completion runs (5 warmup, concurrency 1, `max_tokens` 48) against `https://api.yvr-02.edge.polargrid.ai`, captured 2026-08-19 from a macOS host over the public internet, paced at 1.2 req/s to stay under the per-key rate limit. Server-only is read from the gateway's `pg_metadata` SSE event (`inference_ttft_ms`); end-to-end is client wall-clock. Reasoning mode off (default). CUDA graphs on (`enforce_eager=false`). Raw runs: `benchmarks/fleet-2026-08-18-qwen-3.8/yvr-02-c1.json`.*
**Read the server-only row, not the end-to-end row, when comparing against the [Qwen3.5 27B card](/models/qwen-3.5-27b).** The two captures ran from different client locations, so their network legs differ (210 ms here vs 106 ms there) even though server-side latency is effectively identical. The end-to-end number tells you what *that* benchmark host saw, not what the model got slower at.
**Fleet-wide, not one node.** All 13 production nodes were measured (780 runs, zero errors). The eleven 2-GPU nodes land at **87–91 ms** server TTFT p50 with **30.8–31.4 tok/s** decode — a 4 ms spread across eleven geographically separate machines. The two 4-GPU nodes (`yto-01`, `yvr-02`, which also carry telephony/livekit workloads) sit at **72 ms** with **29.4–29.5 tok/s**: lower TTFT, slightly lower decode. That is a machine-class difference, not a per-node regression.
> **Apples-to-apples disclaimer.** Other providers usually publish only their server-side number; comparing it to our **server-only** row is the fair baseline. Our **end-to-end** row is what you'll see from a customer-side request because PolarGrid runs at the edge. The network row above shows exactly how much that's worth in milliseconds.
## How this compares
| Provider | TTFT p50 | Throughput p50 | Source |
| ----------------------------------------- | ------------------------------------------------- | -------------- | ---------------------------------------------------------------------------------------------- |
| **PolarGrid `qwen-3.8-27b` on Blackwell** | **72 ms** server (284 ms e2e from the bench host) | **29.5 tok/s** | this card |
| Claude Sonnet 4.5 | \~1600 ms | 47.6 tok/s | [artificialanalysis.ai](https://artificialanalysis.ai/models/claude-4-5-sonnet) |
| gpt-4o | \~850 ms | 135 tok/s | [artificialanalysis.ai](https://artificialanalysis.ai/models/gpt-4o) |
| Cerebras (specialty silicon) | n/a | \~2100 tok/s | [cerebras.ai](https://www.cerebras.ai/blog/cerebras-inference-3x-faster) |
| Groq with speculative decoding | n/a | \~1665 tok/s | [artificialanalysis.ai](https://artificialanalysis.ai/models/llama-3-3-instruct-70b/providers) |
PolarGrid wins on TTFT against frontier-reasoning providers because of edge proximity — the server does the work in tens of milliseconds and the client is close to it. PolarGrid remains 4 to 70 times behind specialty silicon on raw throughput; RTX 6000 Pro Blackwell workstation FLOPS are below H100 and H200 datacenter FLOPS. CUDA graphs are enabled (`enforce_eager=false`, the fleet default).
**Against the Qwen3.5 27B it replaced, the swap is latency-neutral at concurrency 1 and better under load.** Measured on the staging canary 2026-08-17: 88 / 121 / 118 / 143 ms at c=1 / 8 / 16 / 32 for 3.8, versus 88 / 129 / 144 / 173 ms for 3.5. Same first-token latency for a single caller; the gap opens as concurrency rises.
## Quickstart
```bash cURL theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Say hi in one short sentence."}],
"stream": true,
"max_tokens": 32
}'
```
```typescript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
for await (const chunk of client.chatCompletionStream({
model: "qwen-3.8-27b",
messages: [{ role: "user", content: "Say hi in one short sentence." }],
maxTokens: 32,
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
}
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
async for chunk in client.chat_completion_stream({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Say hi in one short sentence."}],
"max_tokens": 32,
}):
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
## Capabilities
| Field | Value |
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| Context window | 262,144 tokens (256K) |
| Streaming | Yes (SSE via `stream: true`) |
| Function calling / tools | Yes (Hermes-style; see "Function calling" below) |
| Structured output (`response_format`) | Yes — `json_object` and `json_schema` (vLLM `structured_outputs` constrained decoding) |
| Logprobs | No (vllm\_backend exposes only `text_output` over Triton; not surfaced) |
| Sampling controls | `temperature`, `top_p`, `top_k`, `min_p`, `frequency_penalty`, `presence_penalty`, `repetition_penalty`, `seed`, `stop` |
| Reasoning ("thinking") mode | Off by default; opt in via `"enable_thinking": true` in the request body |
## Function calling
Pass OpenAI-shape `tools` and the model returns a `tool_calls` array on the assistant message (or as a `delta.tool_calls` chunk when streaming). The gateway speaks Qwen's Hermes tool-call template under the hood.
```bash cURL theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Whats the weather in Tokyo?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"tool_choice": "auto"
}'
```
```typescript JavaScript theme={null}
const reply = await client.chatCompletion({
model: "qwen-3.8-27b",
messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
tools: [{
type: "function",
function: {
name: "get_weather",
description: "Get the current weather in a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"]
}
}
}],
tool_choice: "auto",
});
const call = reply.choices[0].message.tool_calls?.[0];
// call.function.name === "get_weather"
// JSON.parse(call.function.arguments) === { city: "Tokyo" }
```
`tool_choice` accepts `"auto"` (model decides), `"none"` (force plain text), `"required"` (force a tool call), or `{ "type": "function", "function": { "name": "" } }` to force a specific tool.
After invoking the tool yourself, append a `role: "tool"` message containing the result and re-call the model:
```json theme={null}
{
"model": "qwen-3.8-27b",
"messages": [
{"role": "user", "content": "Weather in Tokyo?"},
{"role": "assistant", "tool_calls": [
{"id": "call_abc", "type": "function",
"function": {"name": "get_weather", "arguments": "{\"city\":\"Tokyo\"}"}}
]},
{"role": "tool", "tool_call_id": "call_abc",
"content": "{\"temp_c\": 22, \"sky\": \"sunny\"}"}
]
}
```
## Structured output (JSON mode)
Use `response_format` to force the model to emit valid JSON. Backed server-side by vLLM's `structured_outputs` constrained decoding, so the output is guaranteed to parse.
```bash theme={null}
curl https://api.yto-01.edge.polargrid.ai/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Give me a JSON object describing a cat."}],
"response_format": {
"type": "json_schema",
"json_schema": {
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"age_years": {"type": "integer"},
"color": {"type": "string"}
},
"required": ["name", "age_years", "color"]
}
}
}
}'
```
`{"type": "json_object"}` accepts any valid JSON; `json_schema` constrains it to your schema.
### Reasoning mode
Qwen3.8 ships with a "thinking" mode that emits a `...` reasoning trace before the user-visible answer. PolarGrid's `/v1/chat/completions` endpoint disables this by default to keep first-token latency low. The 27B variant runs the same toggle, with deeper reasoning quality at the cost of longer generation time.
To enable thinking on a per-request basis:
```json theme={null}
{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "..."}],
"enable_thinking": true
}
```
## Model identifier
Call this model with the canonical id `qwen-3.8-27b` at all inference endpoints (`/v1/chat/completions`, `/v1/completions`). The HuggingFace repo id `Qwen/Qwen3.8-27B-FP8` is accepted at `/v1/models/load` for hot-loading purposes but does **not** resolve at inference time — use the canonical id for chat and completions calls.
## Notes
* License: [Apache 2.0](https://huggingface.co/Qwen/Qwen3.8-27B-FP8/blob/main/LICENSE) (no auth required to pull weights).
* Native FP8 — no runtime quantization step at load.
* VRAM is tight: a single 46 GB L40S can host this model **or** the voice stack, not both. Multi-GPU edges pin 27B to its own GPU; see `backend/edge-production-setup/CLAUDE.md` for the layout matrix.
## See also
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models
* [`/v1/chat/completions`](/api-reference/chat-completions) — endpoint reference
# HumeAI TADA 3B ML
Source: https://polargrid.mintlify.app/models/tada-3b-ml
Multilingual streaming TTS with cross-lingual voice cloning
HumeAI TADA 3B ML (`tada-3b-ml`) is a multilingual text-to-speech model served on PolarGrid edge nodes via Triton's `python` backend. Unlike preset-voice models, TADA clones a speaker from a short reference clip and can carry that voice across languages — synthesize French in a voice you only ever recorded speaking English.
* **HF repo:** [`HumeAI/tada-3b-ml`](https://huggingface.co/HumeAI/tada-3b-ml)
* **Modality:** Text-to-Speech (streaming)
* **Backend:** Triton `python` (isolated TADA pod)
- **Available regions:** all regions — see [Model availability](/guides/model-availability)
## Headline benchmark
TADA exposes a chunked-HTTP streaming transport for `/v1/audio/speech`.
| Measurement | p50 | p95 |
| -------------------------------------------------- | ---------- | ------- |
| **End-to-end TTFA (with network)** | **352 ms** | 449 ms |
| **Server-only TTFA (gateway → first triton byte)** | **238 ms** | 282 ms |
| *Network leg of TTFA (e2e − server)* | *109 ms* | 210 ms |
| Full-utterance latency (TTLB, client wall-clock) | 919 ms | 1300 ms |
| Real-time factor (RTF) | 0.16 | 0.36 |
*Streaming `/v1/audio/speech` (`stream: true`, `pcm`), 100 runs against
`https://api.yvr-02.edge.polargrid.ai`, captured 2026-05-27 from a
Vancouver-area laptop over the public internet. TTFA is the time to the
first audio byte arriving at the client — identical to TTFB since the
response body is raw PCM. Server-only TTFA comes from the
`X-Pg-First-Byte-Ms` response header
([PR #507](https://github.com/PolarGrid-AI/polargrid-monorepo/pull/507)),
which the gateway sets to its measured time from `_stream_tts` entry to
the first PCM chunk returned by triton. Total synthesis time (TTLB) is
not exposed via header — response headers leave the wire before
synthesis completes — so TTLB stays a client-side wall-clock number. RTF
\= synthesis wall-clock ÷ audio duration; below 1.0 is faster than real
time. 92/100 runs returned a `streaming` verdict; the remainder
finished too fast for the incremental-arrival heuristic to fire, which
is a property of the heuristic, not server-side buffering. Raw runs:
[`benchmarks/yvr-02-2026-05-27/tada-3b-ml/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-05-27/tada-3b-ml/tada-3b-ml_bench.json).
Harness:
[`bench/tada-3b-ml/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/tada-3b-ml).*
## How this compares
| Provider | TTFA p50 | Notes | Source |
| --------------------------------------- | ---------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------- |
| **PolarGrid `tada-3b-ml` on Blackwell** | **352 ms** e2e / **238 ms** server | Multilingual + cross-lingual voice cloning | this card |
| ElevenLabs Turbo v2 | \~200 to 300 ms model / \~478 ms real-world streaming TTFB | English-leaning | [elevenlabs.io](https://elevenlabs.io/docs/eleven-api/concepts/latency) |
| Cartesia Sonic | 90 ms marketing claim / \~188 ms independent p50 | English-leaning, no cross-lingual cloning | [cartesia.ai](https://cartesia.ai/sonic) |
| ElevenLabs Flash | 75 ms marketing claim / \~288 ms independent p50 | English-leaning, no cross-lingual cloning | [gradium.ai](https://gradium.ai/content/best-low-latency-tts-apis-2026) |
| Hume Octave 2 | \~100 to 200 ms TTFT | Hume's newer TTS, would land below TADA | [dev.hume.ai](https://dev.hume.ai/docs/text-to-speech-tts/overview) |
PolarGrid's 238 ms server TTFA is in range of real-world ElevenLabs Turbo v2 streaming TTFB. Cartesia Sonic and ElevenLabs Flash report lower marketing numbers and similar real-world numbers, but ship smaller English-leaning models without cross-lingual cloning, so the comparison is not like for like. Hume Octave 2 has moved the goalpost on Hume's own product line; PolarGrid hosts TADA (the prior generation) faster than Hume hosted it.
## Quickstart
Edge endpoints accept your raw `pg_*` API key as a bearer token — no token exchange. See [Authentication](/authentication). Replace `` with your edge region, or discover the nearest one via the [autorouter](/api-reference/overview#picking-a-region).
```bash cURL theme={null}
curl -X POST https://api..edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer $POLARGRID_API_KEY" \
-H "Content-Type: application/json" \
--no-buffer \
-d '{
"model": "tada-3b-ml",
"input": "Hello from PolarGrid.",
"voice": "default",
"response_format": "pcm",
"stream": true
}' \
--output speech.pcm
```
```javascript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
for await (const chunk of client.textToSpeechStream({
model: "tada-3b-ml",
input: "Hello from PolarGrid.",
voice: "default",
responseFormat: "opus",
})) {
audioPlayer.appendChunk(chunk);
}
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
async for chunk in client.text_to_speech_stream({
"model": "tada-3b-ml",
"input": "Hello from PolarGrid.",
"voice": "default",
"response_format": "opus",
}):
audio_player.append_chunk(chunk)
```
## Capabilities
| Field | Value |
| --------------- | -------------------------------------------------------------------------------------------------------------------------- |
| Endpoint | `POST /v1/audio/speech` |
| Audio output | 24 kHz, 16-bit, mono |
| Streaming | Yes — chunked HTTP, `pcm` and `opus` (`stream: true`); audio delivered incrementally in \~4-token windows during synthesis |
| Batch formats | `pcm`, `wav`, `mp3` |
| Voice model | Cross-lingual voice cloning from a reference clip (no preset voice catalog) |
| Languages | English, French, German, Spanish, Italian, Portuguese, Polish, Japanese, Arabic, Chinese |
| `speed` control | Batch only — streaming requires `speed = 1.0` |
| Max batch size | 1 |
## Voices — cross-lingual cloning
TADA does **not** expose preset voice IDs. The `voice` parameter selects a reference speaker:
| `voice` value | Meaning |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `default` | The bundled reference clip — a neutral English speaker. Use this when you just want speech and don't care about the timbre. |
| A URL | A WAV file (24 kHz mono) fetched and used as the reference. Pair it with `voice_transcript` — the exact text spoken in the clip. |
| A base64 WAV | The reference clip inlined as a base64-encoded WAV string. Also pair with `voice_transcript`. |
`voice_transcript` is required whenever `voice` is a URL or base64 clip — TADA conditions on both the reference audio and its transcript. It is not needed for `voice: "default"`.
The cloned voice carries across languages: provide an English reference clip and set the `language` field (or write the `input` in the target language) to synthesize that speaker in French, Japanese, and so on.
```bash theme={null}
curl -X POST https://api..edge.polargrid.ai/v1/audio/speech \
-H "Authorization: Bearer $POLARGRID_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "tada-3b-ml",
"input": "Bonjour, ceci est une voix clonée.",
"voice": "https://example.com/my-reference.wav",
"voice_transcript": "This is the exact text spoken in the reference clip.",
"language": "fr",
"response_format": "wav"
}' \
--output cloned.wav
```
## Streaming
Pass `stream: true` for chunked audio over a single HTTP response. Streaming formats are `pcm` (default for raw HTTP callers) and `opus`; the PolarGrid SDKs default streaming requests to `opus`. Requesting `wav` or `mp3` with `stream: true` returns `400 Bad Request`.
**`speed` must be 1.0 in streaming mode.** TADA streaming does not support `speed` values other than `1.0`. Setting any other value (e.g., `speed: 1.5`) with `stream: true` returns a `400 Bad Request` error from the gateway. Use batch mode (`stream: false`) if you need speed control.
Audio is delivered incrementally: TADA's synthesis loop runs one step per text token, and the handler emits a chunk every couple of tokens (a \~4-token window, 2 of them overlap for crossfade context) as synthesis proceeds — so the first audio arrives well before the utterance finishes. Synthesis is fast on top of that (real-time factor \~0.13-0.25, i.e. audio produced several times faster than it plays). The `streaming_verdict` field in [`bench/tada-3b-ml/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/tada-3b-ml) reports per run whether the edge delivered bytes incrementally.
TADA streaming does **not** honor the `speed` parameter -- speed control needs a full second synthesis pass, which is incompatible with per-chunk streaming. Pass `speed: 1.0` (or omit it) for streaming requests; use batch mode if you need to change the rate.
See the [Text-to-Speech API reference](/api-reference/text-to-speech#streaming) for the full streaming contract — response headers, truncated-stream detection, and the per-format table.
## Aliases
The following caller-facing aliases resolve to `tada-3b-ml`:
| Alias | Resolves to |
| ----------------- | ------------ |
| `humane-tada` | `tada-3b-ml` |
| `humane/tada-tts` | `tada-3b-ml` |
## Model identifier
Call this model with the canonical id `tada-3b-ml` (or an alias above) at `/v1/audio/speech`. The HuggingFace repo id `HumeAI/tada-3b-ml` is accepted at `/v1/models/load` for hot-loading but does not resolve at inference time.
## Input length limit
`tada-3b-ml` accepts at most **850 characters** of `input` per request. The cap is enforced at the gateway before synthesis; over-limit requests return `413 Payload Too Large` (`Input too long: maximum 850 characters for tada-3b-ml`). The count is taken after surrounding quotes and code/markdown artifacts are stripped, i.e. the text actually synthesized.
The limit is lower than other TTS models (`kokoro-82m` allows 4096) because longer inputs can exhaust GPU memory mid-synthesis. The fixed cap keeps the limit deterministic regardless of server load — without it, the same request could succeed or fail depending on the node's GPU memory state. Split longer text into multiple requests and concatenate the audio client-side.
## Deterministic output
TADA is a diffusion-based TTS model. The inference code seeds the RNG to a fixed value before every generation, so the same `input` text + same `voice` reference produces byte-identical audio across requests. This is intentional: a fixed seed guarantees consistent voice identity and timing, which is important for voice-agent pipelines where unpredictable prosody shifts between calls would degrade the user experience.
Key details:
* **No caching involved.** Each request runs full diffusion inference. Billing applies per request regardless of output similarity.
* **Applies to both batch and streaming modes.** The determinism holds whether you call with `stream: true` or `stream: false`.
* **Planned: user-controllable seed.** A future API version will expose a `seed` parameter so callers can introduce deliberate prosody variation when desired.
## Notes
* TADA runs in its own Triton pod, isolated from the voice pod: hume-tada pins `transformers < 5` and `torch < 2.8`, while the voice pod's `cohere-transcribe` needs `transformers >= 5.4`. See [`backend/edge-production-setup/CLAUDE.md`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/CLAUDE.md) for the pod layout.
* Streaming synthesis is per-token through a decoupled Triton transaction policy — the handler pushes PCM windows as they are decoded rather than buffering the full utterance.
* For preset-voice English/British TTS with a fixed catalog, use [`kokoro-82m`](/models/kokoro-82m) instead — TADA is the choice when you need a specific cloned voice or a non-English language.
## See also
* [Text-to-Speech API](/api-reference/text-to-speech) — endpoint reference, formats, streaming contract
* [Voice AI guide](/guides/voice) — building voice agents on PolarGrid
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models
# Whisper Large V3 Turbo
Source: https://polargrid.mintlify.app/models/whisper-large-v3-turbo
Fast multilingual speech-to-text on PolarGrid edge nodes
Whisper Large V3 Turbo (`whisper-large-v3-turbo`) is OpenAI's 809M-parameter Whisper model, served on PolarGrid edge nodes via Triton's `python` backend using the [faster-whisper](https://github.com/SYSTRAN/faster-whisper) runtime. It is co-resident on the voice pod alongside [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) and the `kokoro-82m` TTS model — see [`backend/edge-production-setup/CLAUDE.md`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/CLAUDE.md) for the pod split rationale.
* **HF repo:** [`openai/whisper-large-v3-turbo`](https://huggingface.co/openai/whisper-large-v3-turbo)
* **Modality:** Speech-to-Text (streaming + sync)
* **Backend:** Triton `python` (voice pod)
* **Parameters:** 809M
- **Available regions:** all regions except `dfw-02` — see [Model availability](/guides/model-availability)
Re-benched 2026-06-11 on the same node, harness, and input corpus as the
previous run. Server-only inference dropped **287 → 145 ms p50** (RTF
**0.050 → 0.025**, \~40× real-time) after the in-process audio resampling fix
on the STT hot path ([PR #612](https://github.com/PolarGrid-AI/polargrid-monorepo/pull/612)) —
the gateway previously paid an out-of-process resample on every
non-16 kHz upload. Streaming time-to-done improved **639 → 582 ms p50**;
TTFT unchanged at \~78 ms. Raw runs:
[`benchmarks/yvr-02-2026-06-11/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/benchmarks/yvr-02-2026-06-11).
## Headline benchmark
`POST /v1/audio/transcriptions?stream=true` is the live streaming surface. The server emits a `text/event-stream` of `transcript.text.delta` events as each decode window completes, then closes with a single `transcript.text.done` event. Consumers can render the rolling transcript immediately instead of waiting for the final result.
| Measurement | p50 | p95 |
| --------------------------------------------------- | --------- | ------- |
| **TTFT (response headers → first non-empty delta)** | **78 ms** | 83 ms |
| Time to `done` event (response headers → done) | 582 ms | 921 ms |
| Interim work (first delta → done) | 504 ms | 841 ms |
| Delta events per request | 4 | 7 (max) |
| Partial deltas (text != final) | 4 | — |
| e2e total (POST → `[DONE]`) | 1304 ms | 1994 ms |
| **RTF (e2e ÷ audio duration)** | **0.23** | 0.30 |
| well\_formed | 100 / 100 | — |
*Bench: 100 streaming transcription runs against `https://api.yvr-02.edge.polargrid.ai`, captured 2026-06-11 from a Vancouver-area laptop. Inputs were the same 5 short utterances (4.2 – 7.7 s, 24 kHz mono WAV, \~200–350 KB each) used by the cohere bench, pre-synthesized via `tada-3b-ml` on the same node. Raw runs: [`benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/whisper-large-v3-turbo_streaming_bench.json).*
> **Delta cadence is chunk-driven.** 4.2 – 5.9 s clips emit 4 deltas, 7.2 – 7.7 s clips emit 7. Deltas are provisional display hints — a delta carries either the new suffix or a full revised hypothesis, and the two are not distinguishable from the event alone, so do not concatenate them; take the final transcript from `done.text` (see [Delta semantics](/api-reference/speech-to-text#delta-semantics)). The first delta arriving at 78 ms after response headers is the meaningful TTFT for live-captioning pipelines. The gateway does not emit `X-Pg-Inference-Ms` on the stream surface yet, so server-only inference cannot be quoted client-side for streaming today.
## Live WebSocket benchmark
`wss://…/v1/audio/transcriptions/ws` transcribes **while audio is being
captured**: the client streams 16 kHz mono PCM frames at microphone pace and
partial transcripts arrive during the utterance — no more waiting for the
upload to finish. Benched 2026-06-11 with the same five clips streamed in
100 ms frames at real-time pace, 100 runs after a 5-run warmup. E2e is
bounded by the audio duration by design; the numbers that matter are the
partial cadence and the end-of-speech lag:
| Metric | p50 | p95 |
| -------------------------------------------------------- | ---------- | ------------------------ |
| First partial (first frame sent → first non-empty delta) | 1023 ms | 1042 ms |
| Partials while audio still flowing | 4 | 7 (max, 7.2–7.7 s clips) |
| **`stop` → authoritative `done`** | **159 ms** | 205 ms |
| well\_formed | 100 / 100 | — |
Raw runs: [`whisper-large-v3-turbo/whisper-large-v3-turbo_ws_bench.json`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/whisper-large-v3-turbo_ws_bench.json).
Harness: [`bench/whisper-large-v3-turbo/bench_ws.py`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/backend/edge-production-setup/bench/whisper-large-v3-turbo/bench_ws.py).
The first partial lands at the \~1 s window boundary (partials are emitted per
second of new audio) and every run produced partials while audio was still
arriving. The **159 ms p50 stop→done** figure is the effective
end-of-speech-to-transcript latency — the number to compare against
streaming STT providers. The upload surfaces instead pay full-clip
transcription after the audio ends (\~580 ms to done on the SSE surface).
Sessions cap at 120 s of audio; wire protocol and delta semantics in the
[Speech-to-Text API reference](/api-reference/speech-to-text#live-streaming--websocket).
## How this compares
Same node, same inputs, same client as the [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) benches:
| Model | Streaming first-partial p50 | ttft\_done p50 | e2e total p50 | RTF (e2e) |
| ---------------------------- | --------------------------- | -------------- | ------------- | --------- |
| `cohere-transcribe-03-2026` | **44 ms** | **560 ms** | 2510 ms | 0.42 |
| **`whisper-large-v3-turbo`** | 77 ms | 639 ms | **1473 ms** | **0.25** |
Cohere reaches its first partial sooner; whisper finishes the full streamed transcript materially faster end-to-end (1473 vs 2510 ms p50) at a lower real-time factor. Pick cohere for earliest-possible interim display and broad accented-language accuracy; pick whisper for fastest full-utterance completion.
*Both rows are from the pre-PR-#612 runs (2026-05-28 / 2026-06-02) so they remain apples-to-apples. Whisper's post-fix numbers are lower (see the headline table above); cohere has not yet been re-benched on the fixed hot path.*
## Quickstart
Edge endpoints accept your raw `pg_*` API key as a bearer token — no token exchange. See [Authentication](/authentication).
```bash cURL theme={null}
curl -X POST "https://api.yvr-02.edge.polargrid.ai/v1/audio/transcriptions?sync=true&model=whisper-large-v3-turbo" \
-H "Authorization: Bearer $POLARGRID_API_KEY" \
-F "file=@input.wav"
```
```typescript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";
import fs from "node:fs";
const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });
const result = await client.audioTranscriptions({
model: "whisper-large-v3-turbo",
file: fs.createReadStream("input.wav"),
// Default mode is async (returns job_id). Pass sync to block on the result.
sync: true,
});
console.log(result.text);
```
```python Python theme={null}
from polargrid import PolarGrid
client = await PolarGrid.create(api_key="pg_...")
with open("input.wav", "rb") as f:
result = await client.audio_transcriptions({
"model": "whisper-large-v3-turbo",
"file": f,
"sync": True,
})
print(result["text"])
```
## Endpoint modes
`POST /v1/audio/transcriptions` has three upload modes selected by query params (not multipart fields), plus a live WebSocket surface:
| Mode | Surface | Response | Use when |
| -------------------- | ------------------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Live (WebSocket)** | `wss://…/v1/audio/transcriptions/ws` | `transcript.text.delta` events **while audio is still arriving**, `done` after `stop` | Voice agents and live captioning from a microphone — partials during speech, final transcript \~160 ms after end-of-speech. |
| **Streaming** | `?stream=true` | `text/event-stream` of delta + done events | Rolling display over a complete file upload. |
| **Sync** | `?sync=true` | `200` with the formatted transcript directly | Voice-agent path that needs the answer in one round trip. |
| **Async (default)** | *none* | `202` with `{job_id, poll_url}`; poll via `GET ?job_id=...` | Background batch transcription; no caller blocking. |
`stream` and `sync` are mutually exclusive — passing both returns `400`. Streaming requires `response_format` in `{json, text}`. The WebSocket surface takes 16 kHz mono int16 PCM frames and caps sessions at 120 s — see the [Speech-to-Text API reference](/api-reference/speech-to-text#live-streaming--websocket) for the wire protocol and delta semantics.
### Sync benchmark (`?sync=true`)
The blocking surface holds the connection open until inference completes, then ships the whole JSON response. Client-side TTFB ≈ total wall-clock, and the meaningful split is **server inference time** vs **network leg** (which for STT is dominated by the audio upload). Raw runs: [`benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yvr-02-2026-06-11/whisper-large-v3-turbo/whisper-large-v3-turbo_bench.json).
| Measurement | p50 | p95 |
| --------------------------------------- | ---------- | ------- |
| End-to-end TTFB (with network + upload) | 909 ms | 1329 ms |
| **Server-only inference** | **145 ms** | 176 ms |
| *Network leg (e2e − server)* | *765 ms* | — |
| Body transfer (JSON response) | 1.1 ms | 2.4 ms |
| RTF (server inference ÷ audio duration) | 0.025 | 0.031 |
Server-only timing comes from the `X-Pg-Inference-Ms` response header (PR #507), available on the sync surface.
> **The network leg is upload-dominated.** Each request ships a multi-second WAV file before inference can begin, so the 765 ms p50 network figure is largely the upload time of a \~300 KB body — not POP-to-client RTT. For shorter clips (sub-2 s) the network leg shrinks proportionally. Quote the **server-only RTF** when comparing inference throughput against centralized providers.
## Capabilities
| Field | Value |
| ----------------- | ------------------------------------------------------------------------------------------------ |
| Endpoint | `POST /v1/audio/transcriptions` |
| Multipart field | `file` (required) |
| Query params | `model`, `language`, `prompt`, `temperature`, `response_format`, `punctuation`, `stream`, `sync` |
| `response_format` | `json` (default), `text`, `srt`, `vtt`, `verbose_json` |
| Languages | Multilingual — auto-detected, or pin with `language` (e.g. `en`, `fr`, `es`) |
| Max batch size | 1 |
| Backend pod | `inference-backend-triton-voice` |
## Response timing headers
PR #507 added two response headers that bench harnesses and observability tooling can read to get a server-only inference time without inferring it from the body:
| Header | Value |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `X-Pg-Inference-Ms` | Integer milliseconds — wall-clock around the `transcribe()` call inside the gateway's `_handle_sync` (i.e., excludes the audio upload + response transfer). |
| `Server-Timing` | Standard-shape `inference;dur=` entry carrying the same number. |
These let callers compute the network leg as `(client wall-clock) − (X-Pg-Inference-Ms)`, the same e2e-vs-server split available for LLM via `pg_metadata`.
## Model identifier
Call this model with the canonical id `whisper-large-v3-turbo` at `/v1/audio/transcriptions`. It has no short alias. The HuggingFace repo id `openai/whisper-large-v3-turbo` is accepted at `/v1/models/load` for hot-loading purposes but does not resolve at inference time.
## Notes
* License: [Apache 2.0](https://huggingface.co/openai/whisper-large-v3-turbo) (no auth required to pull weights).
* Runtime: served via `faster-whisper`, not raw `transformers`, for optimized CTranslate2 inference.
* For multilingual coverage and accuracy on accented speech, [`cohere-transcribe-03-2026`](/models/cohere-transcribe-03-2026) is the alternative — whisper-turbo's edge is lowest full-utterance latency.
## See also
* [Speech-to-Text API](/api-reference/speech-to-text) — endpoint reference, formats, streaming contract
* [Voice AI guide](/guides/voice) — building voice agents on PolarGrid
* [Authentication](/authentication) — using your `pg_*` API key
* [`/v1/models`](/api-reference/models) — list all available models
# Pricing
Source: https://polargrid.mintlify.app/pricing
Simple, transparent pricing for edge AI inference
# Pricing
Pay only for what you use. No upfront commitments, no minimum fees.
Every new account gets **\$500 in free credits** to get started.
## LLM Inference
| Model | Input (per 1M tokens) | Output (per 1M tokens) |
| ------------ | --------------------- | ---------------------- |
| Qwen 3.8 27B | \$0.20 | \$0.75 |
| Qwen 3.6 35B | \$0.20 | \$0.96 |
## Audio
| Service | Model | Rate |
| -------------- | ---------------------- | ------------- |
| Speech-to-Text | Whisper Large V3 Turbo | \$0.004 / min |
| Speech-to-Text | Cohere Transcribe | \$0.004 / min |
| Text-to-Speech | Hume AI TADA | \$0.009 / min |
| Text-to-Speech | Kokoro 82M | \$0.006 / min |
## Voice Pipeline
| Service | Rate |
| ----------------------------- | ------------- |
| Voice Agent (STT + LLM + TTS) | \$0.070 / min |
| PersonaPlex (Voice-to-Voice) | \$0.070 / min |
## Volume Discounts
Committed monthly usage unlocks automatic discounts:
| Monthly Commitment | Discount |
| ------------------- | --------------------------------------- |
| \$5,000 - \$9,999 | 5% |
| \$10,000 - \$19,999 | 10% |
| \$20,000 - \$40,000 | 15% |
| \$40,000+ | [Contact us](mailto:hello@polargrid.ai) |
## Billing Details
| | |
| ----------------- | ------------------------------------------------------------------------------------------ |
| **Free trial** | \$500 in credits on signup |
| **Billing cycle** | Calendar month |
| **Payment** | Credit card via Stripe |
| **Metering** | Pay-as-you-go, billed by actual usage |
| **Invoices** | Auto-generated monthly |
| **Dashboard** | Real-time usage tracking at [app.polargrid.ai](https://app.polargrid.ai/dashboard/billing) |
## Enterprise
Need custom pricing, volume allocations, or a design partner arrangement? Contact us at [hello@polargrid.ai](mailto:hello@polargrid.ai).
Make your first API call in 5 minutes
See all available models and capabilities
# Quickstart
Source: https://polargrid.mintlify.app/quickstart
Get your first API call working in 5 minutes
# Quickstart
This guide will get you from zero to your first PolarGrid API call in under 5 minutes.
## 1. Get an API Key
Create an account at app.polargrid.ai
Go to Settings → API Keys and click **Generate New Key**
Your key starts with `pg_`. Keep it secure — you won't see it again.
## 2. Make Your First Request
```bash cURL theme={null}
# Set your API key
export API_KEY="pg_your_api_key"
# Ask the autorouter which edge is fastest for the caller — it returns
# {region, name, endpoint, ttl}. The endpoint is the actual base URL for
# inference. Autorouter only serves /v1/route; it does NOT proxy inference.
EDGE=$(curl -s https://autorouter.polargrid.ai/v1/route | jq -r .endpoint)
# List available models on that edge — send the API key directly.
curl -s "$EDGE/v1/models" -H "Authorization: Bearer $API_KEY" | jq '.data[].id'
# Run a chat completion against the same edge
curl -s -X POST "$EDGE/v1/chat/completions" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}]
}' | jq
```
```javascript JavaScript theme={null}
import { PolarGrid } from '@polargrid/polargrid-sdk';
// Auto-select fastest region and handle auth automatically
const client = await PolarGrid.create({
apiKey: 'pg_your_api_key',
});
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [
{ role: 'user', content: 'Hello!' }
]
});
console.log(response.choices[0].message.content);
```
```python Python theme={null}
from polargrid import PolarGrid
# Auto-select fastest region and handle auth automatically
client = await PolarGrid.create(api_key="pg_your_api_key")
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [
{"role": "user", "content": "Hello!"}
]
})
print(response.choices[0].message.content)
```
The edge accepts your `pg_*` API key directly. The SDKs additionally handle region selection automatically. See [Authentication](/authentication) for details.
## 3. Choose Your Region
PolarGrid automatically routes to the fastest region when you use `PolarGrid.create()`. You can also specify one explicitly:
```javascript JavaScript theme={null}
// Auto-select best region (recommended)
const client = await PolarGrid.create({ apiKey: 'pg_...' });
console.log(`Connected to: ${client.getRegionName()}`); // e.g., "Toronto"
// Or specify a region explicitly
const client = new PolarGrid({
apiKey: 'pg_...',
region: 'toronto' // aliases: 'yto-01', 'yto'
});
```
```python Python theme={null}
# Auto-select best region (recommended)
client = await PolarGrid.create(api_key="pg_...")
print(f"Connected to: {client.get_region_name()}") # e.g., "Toronto"
# Or specify a region explicitly
client = PolarGrid(api_key="pg_...", region="toronto")
```
## Next Steps
Stream tokens as they're generated
Text-to-speech and speech-to-text
Full endpoint documentation
Handle errors gracefully
# CLI Commands
Source: https://polargrid.mintlify.app/sdks/cli/commands
Complete CLI command reference
# CLI Commands
Complete reference for all PolarGrid CLI commands.
## Authentication
```bash theme={null}
polargrid login # Browser OAuth flow
polargrid login --headless # Show CI/CD instructions
polargrid logout # Clear stored credentials
polargrid whoami # Show current user and organization
```
### Login
Opens browser for OAuth authentication:
```bash theme={null}
polargrid login
```
For CI/CD, use environment variable instead:
```bash theme={null}
export POLARGRID_API_KEY="pg_your_key"
```
### Whoami
Shows your current authentication context:
```bash theme={null}
polargrid whoami
```
Output:
```
User
Email: you@example.com
ID: abc123
Current Organization
Name: My Company
ID: org_xyz
Plan: pro
```
## Organizations
```bash theme={null}
polargrid orgs list # List organizations you belong to
polargrid orgs switch # Switch to a different organization
polargrid orgs current # Show current organization
```
### List Organizations
```bash theme={null}
polargrid orgs list
```
Shows all organizations you're a member of, with your role in each.
### Switch Organization
```bash theme={null}
# By ID
polargrid orgs switch org_abc123
# By partial ID
polargrid orgs switch abc
# By name (case-insensitive)
polargrid orgs switch "my company"
# By slug
polargrid orgs switch my-company
```
## API Keys
```bash theme={null}
polargrid keys list # List API keys
polargrid keys create # Create a new API key
polargrid keys revoke # Revoke an API key
```
### Create Key
```bash theme={null}
polargrid keys create my-app-key
polargrid keys create prod-key --permissions admin
polargrid keys create dev-key --permissions read-only
polargrid keys create project-key --project proj_123
```
Options:
* `-p, --permissions `: `read-only`, `read-write`, or `admin` (default: `read-write`)
* `--project `: Scope key to a specific project
Copy your API key immediately — you won't be able to see it again!
### Revoke Key
```bash theme={null}
polargrid keys revoke key_abc123
polargrid keys revoke key_abc123 --yes # Skip confirmation
```
## Inference
### Chat
Interactive streaming chat with edge models:
```bash theme={null}
# Interactive mode (multi-turn conversation)
polargrid chat
# Single-shot mode
polargrid chat "What is the capital of France?"
polargrid chat "Explain quantum computing" --no-interactive
# With options
polargrid chat --model qwen-3.8-27b --region yto-01 --system "You are a helpful assistant"
```
Options:
* `-r, --region `: Edge region
* `-m, --model `: Model (default: `qwen-3.8-27b`)
* `-s, --system `: System prompt
* `-t, --temperature `: Temperature 0.0-2.0 (default: `0.7`)
* `--no-interactive`: Single-shot mode, no follow-up
In interactive mode, type `/exit` or `/quit` to end the conversation.
### Completions
Generate text completions:
```bash theme={null}
polargrid completions "The capital of Canada is"
polargrid completions "Once upon a time" --max-tokens 500 --temperature 0.9
```
Options:
* `-r, --region `: Edge region
* `-m, --model `: Model (default: `qwen-3.8-27b`)
* `--max-tokens `: Maximum tokens (default: `200`)
* `-t, --temperature `: Temperature 0.0-2.0 (default: `0.7`)
### Text-to-Speech
Convert text to audio:
```bash theme={null}
polargrid tts "Hello from PolarGrid"
polargrid tts "Welcome to our platform" --voice af_bella --output welcome.mp3
polargrid tts "Quick announcement" --format wav --speed 1.2
```
Options:
* `-r, --region `: Edge region
* `-m, --model `: TTS model (default: `kokoro-82m`)
* `-v, --voice `: Voice (default: `af_bella`)
* `-f, --format `: `mp3`, `wav`, `opus`, `flac` (default: `mp3`)
* `-o, --output `: Output file (default: `output.`, e.g., `output.mp3`)
* `--speed `: Speed 0.25-4.0 (default: `1.0`)
Available voices: `alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer`, `af_bella`, `af_sarah`, `am_adam`, `am_michael`, `bf_emma`, `bf_isabella`, `bm_george`, `bm_lewis`
### Transcribe
Transcribe audio to text:
```bash theme={null}
polargrid transcribe recording.mp3
polargrid transcribe meeting.wav --language en --format srt --output subtitles.srt
polargrid transcribe audio.mp3 --format verbose_json
```
Options:
* `-r, --region `: Edge region
* `-m, --model `: STT model (default: `whisper-large-v3-turbo`)
* `-l, --language `: Language code (e.g., `en`, `fr`, `es`)
* `-f, --format `: `text`, `json`, `srt`, `vtt`, `verbose_json` (default: `text`)
* `-o, --output `: Output file (default: stdout)
### Translate
Translate audio to English:
```bash theme={null}
polargrid translate french-audio.mp3
polargrid translate interview.wav --format json --output translation.json
```
Options:
* `-r, --region `: Edge region
* `-m, --model `: STT model (default: `whisper-large-v3-turbo`)
* `-f, --format `: `text`, `json` (default: `text`)
* `-o, --output `: Output file (default: stdout)
## Models
Manage models on edge servers. Requires `POLARGRID_API_KEY`.
```bash theme={null}
polargrid models list # List available models
polargrid models status # Show model loading status
```
### List Models
```bash theme={null}
polargrid models list --region yto-01
```
### Model Status
```bash theme={null}
polargrid models status --region yto-01
```
Shows which models are loaded, loading, or failed with colored status indicators.
## GPU
Monitor and manage GPU resources. Requires `POLARGRID_API_KEY`.
```bash theme={null}
polargrid gpu status # GPU utilization and info
polargrid gpu memory # GPU memory usage
```
### GPU Status
```bash theme={null}
polargrid gpu status --region yto-01
```
Shows GPU name, utilization %, memory usage, temperature, and running processes.
### GPU Memory
```bash theme={null}
polargrid gpu memory --region yto-01
```
Visual bar chart of memory usage per GPU.
## Regions
```bash theme={null}
polargrid regions list # List available regions with latency
polargrid regions ping # Ping all regions, show RTT
```
### List Regions
```bash theme={null}
polargrid regions list
```
Shows all regions with current latency from your location.
### Ping Regions
```bash theme={null}
polargrid regions ping
polargrid regions ping --count 5 # 5 pings per region (default: 3)
```
Shows min/avg/max latency to each region.
## Testing
```bash theme={null}
polargrid test # Health check (default)
polargrid test health # Check edge infrastructure health
polargrid test inference # Run a quick inference test
```
### Health Check
```bash theme={null}
# Check all regions
polargrid test health
# Check specific region
polargrid test health --region yto-01
```
### Inference Test
```bash theme={null}
polargrid test inference \
--region yto-01 \
--model qwen-3.8-27b \
--prompt "What is the capital of France?"
```
Options:
* `-r, --region `: Target region (required unless default set)
* `-m, --model `: Model to use (default: `qwen-3.8-27b`)
* `-p, --prompt `: Prompt to send (default: "Hello, how are you?")
## Shell Completions
This is `polargrid completion` (singular) — not to be confused with `polargrid completions` (text inference).
```bash theme={null}
polargrid completion bash # Generate bash completions
polargrid completion zsh # Generate zsh completions
polargrid completion fish # Generate fish completions
```
Install completions for your shell:
```bash theme={null}
# Bash
eval "$(polargrid completion bash)" >> ~/.bashrc
# Zsh
eval "$(polargrid completion zsh)" >> ~/.zshrc
# Fish
polargrid completion fish > ~/.config/fish/completions/polargrid.fish
```
## Configuration
```bash theme={null}
polargrid config list # Show current config
polargrid config get # Get a config value
polargrid config set # Set a config value
polargrid config unset # Remove a config value
```
### Available Config Options
| Key | Description |
| ------------------- | ------------------------------------------ |
| `default_region` | Default edge region for requests |
| `output_format` | Output format: `json`, `table`, or `plain` |
| `api_base_url` | Override API base URL |
| `supabase_url` | Supabase project URL |
| `supabase_anon_key` | Supabase anonymous key |
### Examples
```bash theme={null}
# Set default region
polargrid config set default_region yto-01
# Check default region
polargrid config get default_region
# Clear default region
polargrid config unset default_region
# View all config
polargrid config list
```
## Configuration Files
The CLI stores configuration in `~/.polargrid/`:
| File | Description |
| ----------------- | ------------------------------------------------------------------------------------------------------------------- |
| OS keychain entry | Auth tokens, **encrypted at rest** (macOS Keychain / Windows Credential Manager / Linux Secret Service) |
| `credentials.enc` | AES-256-GCM encrypted token fallback (mode 0600) — used only where no keychain is available, e.g. headless Linux/CI |
| `config.json` | CLI configuration (non-secret) |
Set `POLARGRID_DISABLE_KEYCHAIN=1` to force the encrypted-file backend. An existing plaintext `credentials.json` from an older CLI is migrated into the encrypted store on first use and then deleted.
## CI/CD Example
```bash theme={null}
#!/bin/bash
# ci-test.sh
export POLARGRID_API_KEY="${POLARGRID_API_KEY}"
# Health check
polargrid test health --region yto-01
# Run inference test
polargrid test inference \
--region yto-01 \
--model qwen-3.8-27b \
--prompt "Integration test: respond with OK"
# Single-shot chat
polargrid chat "Summarize: PolarGrid is an edge AI platform" --no-interactive --region yto-01
```
# CLI Quickstart
Source: https://polargrid.mintlify.app/sdks/cli/quickstart
Get started with the PolarGrid CLI
# CLI
Command-line interface for PolarGrid Edge AI Infrastructure.
## Installation
```bash theme={null}
npm install -g @polargrid/cli
```
## Quick Start
```bash theme={null}
# Login to PolarGrid (opens browser)
polargrid login
# Check who you're logged in as
polargrid whoami
# List your organizations
polargrid orgs list
# Create an API key
polargrid keys create my-app-key
# Set default region
polargrid config set default_region yto-01
# Chat with a model (streaming)
polargrid chat "What is the capital of Canada?"
# Text-to-speech
polargrid tts "Hello from PolarGrid" --output hello.mp3
```
## Authentication
### Browser Login (Interactive)
```bash theme={null}
polargrid login
```
Opens your browser for OAuth authentication. Your credentials are stored **encrypted at rest** in your OS keychain (macOS Keychain, Windows Credential Manager, or Linux Secret Service), with an AES-256-GCM encrypted file at `~/.polargrid/credentials.enc` as a fallback where no keychain is available.
### API Key (CI/CD)
For CI/CD pipelines, no login is needed. Just set the environment variable:
```bash theme={null}
export POLARGRID_API_KEY="pg_your_api_key"
# Now you can run inference commands
polargrid chat "Hello" --no-interactive
```
### Check Current Session
```bash theme={null}
polargrid whoami
```
Shows your email, user ID, and current organization.
### Logout
```bash theme={null}
polargrid logout
```
## Shell Completions
Enable tab-completion for commands, subcommands, and flags:
```bash theme={null}
# Bash
eval "$(polargrid completion bash)" >> ~/.bashrc
# Zsh
eval "$(polargrid completion zsh)" >> ~/.zshrc
# Fish
polargrid completion fish > ~/.config/fish/completions/polargrid.fish
```
## Next Steps
Complete command reference for auth, inference, models, GPU, and more
# JavaScript Quickstart
Source: https://polargrid.mintlify.app/sdks/javascript/quickstart
Get started with the PolarGrid JavaScript SDK
# JavaScript SDK
The official JavaScript/TypeScript SDK for PolarGrid.
## Installation
```bash theme={null}
npm install @polargrid/polargrid-sdk
```
## Quick Start
```typescript theme={null}
import { PolarGrid } from '@polargrid/polargrid-sdk';
// Auto-select best region by latency (recommended)
const client = await PolarGrid.create({
apiKey: 'pg_your_api_key',
});
console.log(`Connected to: ${client.getRegionName()}`); // e.g., "Toronto"
// Chat completion
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [
{ role: 'user', content: 'Hello!' }
]
});
console.log(response.choices[0].message.content);
```
## Configuration
```typescript theme={null}
// Sync constructor (uses default region)
const client = new PolarGrid({
// API key (or set POLARGRID_API_KEY env var)
apiKey: 'pg_your_api_key',
// Region alias: 'toronto', 'montreal', 'vancouver', 'new-york', 'dallas', 'san-francisco'
// Or explicit ID: yto-01, yul-01, yvr-02, nyc-01, nyc-02, dfw-01, dfw-02, sfo-01, lax-01, sea-01, chi-01, phx-01, was-01, mia-01, sfo-03
region: 'toronto',
// Request timeout in ms (default: 30000)
timeout: 30000,
// Max retry attempts (default: 3)
maxRetries: 3,
// Enable debug logging
debug: true,
// Use mock data for development
useMockData: false,
});
// Async factory with auto-routing (recommended)
const client = await PolarGrid.create({
apiKey: 'pg_your_api_key',
debug: true,
});
```
## Auto-Routing
The `PolarGrid.create()` factory calls the autorouter (`GET https://autorouter.polargrid.ai/v1/route`), which returns the optimal edge based on your origin:
```typescript theme={null}
const client = await PolarGrid.create({
apiKey: 'pg_your_api_key',
debug: true, // See selected region
});
// [PolarGrid] Auto-routing: selected Toronto (yto-01)
console.log(client.getRegionId()); // 'yto-01'
console.log(client.getRegionName()); // 'Toronto'
```
## Streaming
```typescript theme={null}
for await (const chunk of client.chatCompletionStream({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Tell me a story' }],
})) {
const content = chunk.choices[0]?.delta?.content;
if (content) {
process.stdout.write(content);
}
}
```
## Mock Mode
Perfect for frontend development without a backend:
```typescript theme={null}
const client = new PolarGrid({
useMockData: true, // No API calls, instant realistic responses
debug: true,
});
// All methods work with realistic mock data
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Hello!' }]
});
```
## Environment Variables
```bash theme={null}
POLARGRID_API_KEY=pg_your_api_key
POLARGRID_BASE_URL=https://api.yto-01.edge.polargrid.ai # Optional. Pins the SDK to a specific edge. Leave unset and call PolarGrid.create() to auto-discover the fastest region via https://autorouter.polargrid.ai/v1/route.
```
## Next Steps
Complete API reference
Text-to-speech and speech-to-text
# JavaScript Reference
Source: https://polargrid.mintlify.app/sdks/javascript/reference
Complete JavaScript SDK API reference
# JavaScript SDK Reference
Full API reference for the PolarGrid JavaScript SDK.
## Client Methods
### Text Inference
| Method | Description |
| ------------------------------- | -------------------------------------- |
| `chatCompletion(request)` | Generate chat completion |
| `chatCompletionStream(request)` | Streaming chat completion |
| `completion(request)` | Generate text completion |
| `completionStream(request)` | Streaming text completion |
| `generate(request)` | Legacy generate (wraps chatCompletion) |
### Voice / Audio
| Method | Description |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `textToSpeech(request)` | Convert text to audio (returns ArrayBuffer) |
| `textToSpeechStream(request)` | Streaming TTS over chunked HTTP. Yields `Uint8Array` chunks. Default `responseFormat` is `'opus'`. WebSocket transport is not supported — see the [TTS API streaming section](/api-reference/text-to-speech#streaming). |
| `transcribe(request)` | Transcribe audio to text |
| `transcribeStream(request)` | Streaming transcription. Async iterator of `TranscriptionStreamEvent` (`transcript.text.delta`, `transcript.text.done`, `error`). See the [STT API streaming section](/api-reference/speech-to-text#streaming-transcription-sse). |
| `translate(request)` | Translate audio to English |
### Models
| Method | Description |
| ---------------------- | ---------------------------- |
| `listModels()` | List available models |
| `loadModel(request)` | Load a model into GPU memory |
| `unloadModel(request)` | Unload a model |
| `unloadAllModels()` | Unload all models |
| `getModelStatus()` | Get model loading status |
### GPU
| Method | Description |
| ------------------- | -------------------------- |
| `getGPUStatus()` | Detailed GPU status |
| `getGPUMemory()` | Simplified GPU memory info |
| `purgeGPU(request)` | Clear GPU memory |
### Health & Region
| Method | Description |
| ----------------- | ------------------------------------------- |
| `health()` | Service health check |
| `getRegionId()` | Get current region ID (e.g., 'yvr-02') |
| `getRegionName()` | Get current region name (e.g., 'Vancouver') |
## Error Handling
```typescript theme={null}
import {
PolarGrid,
isPolarGridError,
AuthenticationError,
BillingError,
ValidationError,
RateLimitError,
NetworkError,
TimeoutError,
NotFoundError,
ServerError,
} from '@polargrid/polargrid-sdk';
try {
const response = await client.chatCompletion(request);
} catch (error) {
if (isPolarGridError(error)) {
console.error(`Error: ${error.message}`);
console.error(`Request ID: ${error.requestId}`);
if (error instanceof AuthenticationError) {
// Invalid or expired API key
} else if (error instanceof BillingError) {
// Insufficient credits or payment required (HTTP 402)
} else if (error instanceof ValidationError) {
// Invalid request parameters
console.error('Details:', error.details);
} else if (error instanceof RateLimitError) {
// Rate limited - wait and retry
console.error(`Retry after: ${error.retryAfter}s`);
} else if (error instanceof NetworkError) {
// Network/connection error
} else if (error instanceof TimeoutError) {
// Request timed out
} else if (error instanceof NotFoundError) {
// Resource not found
} else if (error instanceof ServerError) {
// Server error (5xx)
}
}
}
```
## Types
```typescript theme={null}
import type {
// Config
PolarGridConfig,
// Text Inference
ChatCompletionRequest,
ChatCompletionResponse,
ChatCompletionChunk,
CompletionRequest,
CompletionResponse,
CompletionChunk,
GenerateRequest,
GenerateResponse,
// Voice
TextToSpeechRequest,
TranscriptionRequest,
TranscriptionResponse,
VerboseTranscriptionResponse,
TranslationRequest,
TranslationResponse,
TTSVoice,
STTModel,
AudioFormat,
// Models
ModelInfo,
ListModelsResponse,
LoadModelRequest,
LoadModelResponse,
UnloadModelRequest,
UnloadModelResponse,
UnloadAllModelsResponse,
ModelStatusResponse,
// GPU
GPUStatusResponse,
GPUMemoryResponse,
GPUPurgeRequest,
GPUPurgeResponse,
GPUInfo,
GPUMemoryInfo,
GPUProcess,
// Health
HealthResponse,
// Errors
ApiError,
} from '@polargrid/polargrid-sdk';
```
## Regions
```typescript theme={null}
import { POLARGRID_REGIONS } from '@polargrid/polargrid-sdk';
// Available regions
console.log(POLARGRID_REGIONS);
// {
// 'yto-01': { id: 'yto-01', name: 'Toronto', url: 'https://api.yto-01.edge.polargrid.ai' },
// 'yul-01': { id: 'yul-01', name: 'Montreal', url: 'https://api.yul-01.edge.polargrid.ai' },
// 'yvr-02': { id: 'yvr-02', name: 'Vancouver', url: 'https://api.yvr-02.edge.polargrid.ai' },
// 'nyc-01': { id: 'nyc-01', name: 'New York', url: 'https://api.nyc-01.edge.polargrid.ai' },
// 'nyc-02': { id: 'nyc-02', name: 'New York 02', url: 'https://api.nyc-02.edge.polargrid.ai' },
// 'dfw-01': { id: 'dfw-01', name: 'Dallas', url: 'https://api.dfw-01.edge.polargrid.ai' },
// 'dfw-02': { id: 'dfw-02', name: 'Dallas 02', url: 'https://api.dfw-02.edge.polargrid.ai' },
// 'sfo-01': { id: 'sfo-01', name: 'San Francisco', url: 'https://api.sfo-01.edge.polargrid.ai' },
// 'lax-01': { id: 'lax-01', name: 'Los Angeles', url: 'https://api.lax-01.edge.polargrid.ai' },
// 'sea-01': { id: 'sea-01', name: 'Seattle', url: 'https://api.sea-01.edge.polargrid.ai' },
// 'chi-01': { id: 'chi-01', name: 'Chicago', url: 'https://api.chi-01.edge.polargrid.ai' },
// 'phx-01': { id: 'phx-01', name: 'Phoenix', url: 'https://api.phx-01.edge.polargrid.ai' },
// 'was-01': { id: 'was-01', name: 'Washington DC', url: 'https://api.was-01.edge.polargrid.ai' },
// 'mia-01': { id: 'mia-01', name: 'Miami', url: 'https://api.mia-01.edge.polargrid.ai' },
// 'sfo-03': { id: 'sfo-03', name: 'San Francisco', url: 'https://api.sfo-03.edge.polargrid.ai' },
// }
// Region aliases supported: 'toronto', 'yto', 'vancouver', 'yvr', 'montreal', 'yul',
// 'new-york', 'nyc', 'dallas', 'dfw', 'san-francisco', 'sf', 'sfo'
```
## Default Export
```typescript theme={null}
// Named import (recommended)
import { PolarGrid } from '@polargrid/polargrid-sdk';
// Default import also works
import PolarGrid from '@polargrid/polargrid-sdk';
```
# SDKs Overview
Source: https://polargrid.mintlify.app/sdks/overview
Official SDKs for JavaScript, Python, and CLI
# SDKs
PolarGrid provides official SDKs to make integration easy.
For Node.js and browser applications
Async and sync clients with full type hints
Command-line interface for scripting and CI/CD
## Installation
```bash JavaScript theme={null}
npm install @polargrid/polargrid-sdk
```
```bash Python theme={null}
pip install polargrid-sdk
```
```bash CLI theme={null}
npm install -g @polargrid/cli
```
## Features
All SDKs include:
| Feature | JS | Python | CLI |
| ------------------------------ | :-: | :----: | :-: |
| Chat completions | ✅ | ✅ | ✅ |
| Text completions | ✅ | ✅ | — |
| Streaming | ✅ | ✅ | — |
| Text-to-Speech | ✅ | ✅ | — |
| Speech-to-Text | ✅ | ✅ | — |
| Model management | ✅ | ✅ | — |
| GPU management | ✅ | ✅ | — |
| Auto-routing | ✅ | ✅ | ✅ |
| Mock mode | ✅ | ✅ | — |
| Full TypeScript/Pydantic types | ✅ | ✅ | — |
| Async | ✅ | ✅ | — |
| Sync | — | ✅ | ✅ |
## Quick Example
```javascript JavaScript theme={null}
import { PolarGrid } from '@polargrid/polargrid-sdk';
// Auto-select best region
const client = await PolarGrid.create({
apiKey: process.env.POLARGRID_API_KEY,
});
const response = await client.chatCompletion({
model: 'qwen-3.8-27b',
messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(response.choices[0].message.content);
```
```python Python theme={null}
import os
from polargrid import PolarGrid
# Auto-select best region
client = await PolarGrid.create(api_key=os.environ["POLARGRID_API_KEY"])
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}],
})
print(response.choices[0].message.content)
```
```bash CLI theme={null}
export POLARGRID_API_KEY="pg_your_key"
polargrid test inference --region toronto --prompt "Hello!"
```
## Authentication
All SDKs support these authentication methods:
1. **Constructor parameter**: Pass `apiKey` / `api_key` directly
2. **Environment variable**: Set `POLARGRID_API_KEY`
```bash theme={null}
export POLARGRID_API_KEY="pg_your_api_key"
```
Get your API key from the Console.
# Python Quickstart
Source: https://polargrid.mintlify.app/sdks/python/quickstart
Get started with the PolarGrid Python SDK
# Python SDK
The official Python SDK for PolarGrid with async and sync clients.
## Installation
```bash theme={null}
pip install polargrid-sdk
```
## Quick Start
### Async Client (Recommended)
```python theme={null}
import asyncio
from polargrid import PolarGrid
async def main():
# Auto-select best region by latency
client = await PolarGrid.create(api_key="pg_your_api_key")
print(f"Connected to: {client.get_region_name()}") # e.g., "Toronto"
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [
{"role": "user", "content": "Hello!"}
]
})
print(response.choices[0].message.content)
asyncio.run(main())
```
### Sync Client
```python theme={null}
from polargrid import PolarGridSync
client = PolarGridSync(api_key="pg_your_api_key", region="toronto")
response = client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}]
})
print(response.choices[0].message.content)
```
## Configuration
```python theme={null}
# Async client with auto-routing (recommended)
client = await PolarGrid.create(
api_key="pg_your_api_key", # or set POLARGRID_API_KEY env var
debug=True,
)
# Sync constructor with explicit region
client = PolarGrid(
api_key="pg_your_api_key",
# Region alias: 'toronto', 'montreal', 'vancouver', 'new-york', 'dallas', 'san-francisco'
# Or explicit ID: yto-01, yul-01, yvr-02, nyc-01, nyc-02, dfw-01, dfw-02, sfo-01, lax-01, sea-01, chi-01, phx-01, was-01, mia-01, sfo-03
region="vancouver",
# Request timeout in seconds (default: 30.0)
timeout=30.0,
# Max retry attempts (default: 3)
max_retries=3,
# Enable debug logging
debug=True,
# Use mock data for development
use_mock_data=False,
)
```
## Auto-Routing
```python theme={null}
# Calls the autorouter which returns the optimal edge based on your origin
client = await PolarGrid.create(api_key="pg_...", debug=True)
# [PolarGrid] Auto-routing: selected Toronto (yto-01)
print(client.get_region_id()) # 'yto-01'
print(client.get_region_name()) # 'Toronto'
```
## Streaming
```python theme={null}
async for chunk in client.chat_completion_stream({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Tell me a story"}],
}):
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
## Response Metadata (latency)
PolarGrid attaches per-request metadata (region, latency breakdown) under `response.pg_metadata`. It is **optional** — fields may be `None`, and on the non-stream path `latency_ms` is commonly `None`. Always read it through the typed attribute and guard for `None`; never index a top-level `latency_ms` key (it does not exist and raises `KeyError`).
```python theme={null}
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}],
})
text = response.choices[0].message.content
meta = response.pg_metadata
if meta is not None:
print(f"region={meta.region} latency_ms={meta.latency_ms}") # latency_ms may be None
```
Streaming chat chunks (`ChatCompletionChunk`) do **not** carry `pg_metadata` — only the non-stream `chat_completion` response exposes it. If you need region/latency metadata on a streamed request, issue a non-stream call, or read the timing from your own client-side measurements.
## Mock Mode
```python theme={null}
client = PolarGrid(
use_mock_data=True, # No API calls
debug=True,
)
# All methods return realistic mock data
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}]
})
```
## Type Hints
The SDK uses Pydantic models with full type hints:
```python theme={null}
from polargrid.types import (
ChatCompletionRequest,
ChatCompletionResponse,
Message,
)
# Request with typed model
request = ChatCompletionRequest(
model="qwen-3.8-27b",
messages=[Message(role="user", content="Hello!")],
max_tokens=100,
)
# Or use dict (auto-converted)
response = await client.chat_completion({
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Hello!"}],
})
```
## Environment Variables
```bash theme={null}
export POLARGRID_API_KEY=pg_your_api_key
export POLARGRID_BASE_URL=https://api.yto-01.edge.polargrid.ai # Optional. Pins the SDK to a specific edge. Leave unset and call PolarGrid.create() to auto-discover the fastest region via https://autorouter.polargrid.ai/v1/route.
```
## Next Steps
Complete API reference
Text-to-speech and speech-to-text
# Python Reference
Source: https://polargrid.mintlify.app/sdks/python/reference
Complete Python SDK API reference
# Python SDK Reference
Full API reference for the PolarGrid Python SDK.
## Clients
| Class | Description |
| --------------- | -------------------------- |
| `PolarGrid` | Async client (recommended) |
| `PolarGridSync` | Synchronous wrapper |
## Client Methods
### Text Inference
| Method | Description |
| --------------------------------- | ---------------------------------------- |
| `chat_completion(request)` | Generate chat completion |
| `chat_completion_stream(request)` | Streaming chat completion |
| `completion(request)` | Generate text completion |
| `completion_stream(request)` | Streaming text completion |
| `generate(request)` | Legacy generate (wraps chat\_completion) |
### Voice / Audio
| Method | Description |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `text_to_speech(request)` | Convert text to audio (returns bytes) |
| `text_to_speech_stream(request)` | Streaming TTS over chunked HTTP. Async iterator of `bytes` chunks. Default `response_format` is `'opus'`. WebSocket transport is not supported — see the [TTS API streaming section](/api-reference/text-to-speech#streaming). |
| `transcribe(file, request)` | Transcribe audio to text |
| `transcribe_stream(file, request)` | Streaming transcription. Async iterator of `TranscriptionStreamEvent` (`transcript.text.delta`, `transcript.text.done`, `error`). See the [STT API streaming section](/api-reference/speech-to-text#streaming-transcription-sse). |
| `translate(file, request)` | Translate audio to English |
### Models
| Method | Description |
| ----------------------- | ---------------------------- |
| `list_models()` | List available models |
| `load_model(request)` | Load a model into GPU memory |
| `unload_model(request)` | Unload a model |
| `unload_all_models()` | Unload all models |
| `get_model_status()` | Get model loading status |
### GPU
| Method | Description |
| -------------------- | -------------------------- |
| `get_gpu_status()` | Detailed GPU status |
| `get_gpu_memory()` | Simplified GPU memory info |
| `purge_gpu(request)` | Clear GPU memory |
### Health & Region
| Method | Description |
| ------------------- | ----------------------- |
| `health()` | Service health check |
| `get_region_id()` | Get current region ID |
| `get_region_name()` | Get current region name |
## Error Handling
```python theme={null}
from polargrid import (
PolarGrid,
PolarGridError,
AuthenticationError,
BillingError,
ValidationError,
RateLimitError,
NetworkError,
TimeoutError,
NotFoundError,
ServerError,
)
try:
response = await client.chat_completion(request)
except PolarGridError as e:
print(f"Error: {e.message}")
print(f"Request ID: {e.request_id}")
if isinstance(e, AuthenticationError):
# Invalid or expired API key
pass
elif isinstance(e, BillingError):
# Insufficient credits or payment required (HTTP 402)
pass
elif isinstance(e, ValidationError):
# Invalid request parameters
print("Details:", e.details)
elif isinstance(e, RateLimitError):
# Rate limited
print(f"Retry after: {e.retry_after}s")
elif isinstance(e, NetworkError):
# Network/connection error
pass
elif isinstance(e, TimeoutError):
# Request timed out
pass
```
## Types
All types are Pydantic models with full type hints:
```python theme={null}
from polargrid.types import (
# Config
PolarGridConfig,
# Text Inference
ChatCompletionRequest,
ChatCompletionResponse,
ChatCompletionChunk,
CompletionRequest,
CompletionResponse,
CompletionChunk,
GenerateRequest,
GenerateResponse,
Message,
TokenUsage,
# Voice
TextToSpeechRequest,
TranscriptionRequest,
TranscriptionResponse,
VerboseTranscriptionResponse,
TranslationRequest,
TranslationResponse,
TTSVoice,
STTModel,
AudioFormat,
TranscriptionFormat,
# Models
ModelInfo,
ListModelsResponse,
LoadModelRequest,
LoadModelResponse,
UnloadModelRequest,
UnloadModelResponse,
UnloadAllModelsResponse,
ModelStatusResponse,
# GPU
GPUStatusResponse,
GPUMemoryResponse,
GPUPurgeRequest,
GPUPurgeResponse,
GPUInfo,
GPUMemoryInfo,
GPUProcess,
# Health
HealthResponse,
HealthFeatures,
BackendStatus,
)
```
## Enums
```python theme={null}
from polargrid.types import TTSVoice, STTModel, AudioFormat, TranscriptionFormat
# TTS Voices
TTSVoice.ALLOY # "alloy"
TTSVoice.ECHO # "echo"
TTSVoice.NOVA # "nova"
TTSVoice.AF_BELLA # "af_bella" (Kokoro)
# STT Models
STTModel.WHISPER_1 # "whisper-1"
STTModel.WHISPER_LARGE_V3_TURBO # "whisper-large-v3-turbo"
# Audio Formats
AudioFormat.MP3 # "mp3"
AudioFormat.WAV # "wav"
AudioFormat.OPUS # "opus"
# Transcription Formats
TranscriptionFormat.JSON # "json"
TranscriptionFormat.VERBOSE_JSON # "verbose_json"
TranscriptionFormat.SRT # "srt"
TranscriptionFormat.VTT # "vtt"
```
## Regions
```python theme={null}
from polargrid.client import POLARGRID_REGIONS, REGION_ALIASES
# Available regions
print(POLARGRID_REGIONS)
# {
# 'yto-01': {'id': 'yto-01', 'name': 'Toronto', 'url': '...'},
# 'yul-01': {'id': 'yul-01', 'name': 'Montreal', 'url': '...'},
# 'yvr-02': {'id': 'yvr-02', 'name': 'Vancouver', 'url': '...'},
# 'nyc-01': {'id': 'nyc-01', 'name': 'New York', 'url': '...'},
# 'nyc-02': {'id': 'nyc-02', 'name': 'New York 02', 'url': '...'},
# 'dfw-01': {'id': 'dfw-01', 'name': 'Dallas', 'url': '...'},
# 'dfw-02': {'id': 'dfw-02', 'name': 'Dallas 02', 'url': '...'},
# 'sfo-01': {'id': 'sfo-01', 'name': 'San Francisco', 'url': '...'},
# 'lax-01': {'id': 'lax-01', 'name': 'Los Angeles', 'url': '...'},
# 'sea-01': {'id': 'sea-01', 'name': 'Seattle', 'url': '...'},
# 'chi-01': {'id': 'chi-01', 'name': 'Chicago', 'url': '...'},
# 'phx-01': {'id': 'phx-01', 'name': 'Phoenix', 'url': '...'},
# 'was-01': {'id': 'was-01', 'name': 'Washington DC', 'url': '...'},
# 'mia-01': {'id': 'mia-01', 'name': 'Miami', 'url': '...'},
# 'sfo-03': {'id': 'sfo-03', 'name': 'San Francisco', 'url': '...'},
# }
# Aliases: 'toronto', 'yto', 'vancouver', 'yvr', 'montreal', 'yul',
# 'new-york', 'nyc', 'dallas', 'dfw', 'san-francisco', 'sf', 'sfo'
```
## File Handling
The `transcribe()` and `translate()` methods accept multiple file types:
```python theme={null}
from pathlib import Path
# Path object
transcription = await client.transcribe(
file=Path("recording.mp3"),
request={"model": "whisper-1"}
)
# Bytes
with open("recording.mp3", "rb") as f:
audio_bytes = f.read()
transcription = await client.transcribe(
file=audio_bytes,
request={"model": "whisper-1"}
)
# File-like object
with open("recording.mp3", "rb") as f:
transcription = await client.transcribe(
file=f,
request={"model": "whisper-1"}
)
```