> ## Documentation Index
> Fetch the complete documentation index at: https://polargrid.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Qwen3.6 35B-A3B

> Qwen3.6 35B-A3B MoE text LLM on vLLM with native FP8 weights

Qwen3.6 35B-A3B (`qwen-3.6-35b-a3b`) is a Mixture-of-Experts text LLM served on PolarGrid edge nodes via Triton's `vllm_backend`. It has **35 billion total parameters** but activates only **\~3 billion per token** (256 experts, 8 routed + 1 shared active). Weights ship pre-quantized to FP8 (\~35 GB VRAM) and load directly on PolarGrid's Blackwell edge GPUs without runtime requantization.

* **HF repo:** [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8)
* **Modality:** Text LLM (MoE)
* **Backend:** Triton `vllm` (LLM pod)

- **Available regions:** limited availability (customer pilot) — [contact us](https://polargrid.ai/contact) for access

## Headline benchmark

We publish **two** numbers side by side. End-to-end is what your application actually experiences (request → response, network included). Server-only is what the GPU spends on inference (apples-to-apples vs centralized providers' published "inference-only" figures). The gap is the latency PolarGrid's edge PoP eliminates by being close to the caller.

| Measurement | TTFT p50 | TTFT p95 | Throughput p50 |
| - | - | - | - |
| **End-to-end (with network)** | **212 ms** | 238 ms | **15.8 tok/s** |
| **Server-only (no network)** | **163 ms** | 174 ms | — |
| *Network overhead (e2e − server)* | *49 ms* | — | — |

*Bench: 100 streaming chat-completion runs against the Montreal customer-pilot node (since retired from the public fleet), captured 2026-06-09 from yvr-01 (Vancouver) over the public internet. End-to-end is client wall-clock; server-only is read from the gateway's `pg_metadata` SSE event (`inference_ttft_ms` / `inference_total_ms`). Raw runs: [`benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json).*

> **Apples-to-apples disclaimer.** Other providers usually publish only their server-side number; comparing it to our **server-only** row is the fair baseline. Our **end-to-end** row is what you'll see from a customer-side request because PolarGrid runs at the edge — the network row above shows exactly how much that's worth in milliseconds.

## Quickstart

```bash cURL theme={null}
curl https://api.<region>.edge.polargrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Say hi in one short sentence."}],
    "stream": true,
    "max_tokens": 32
  }'
```

```typescript JavaScript theme={null}
import { PolarGrid } from "@polargrid/polargrid-sdk";

const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });

for await (const chunk of client.chatCompletionStream({
  model: "qwen-3.6-35b-a3b",
  messages: [{ role: "user", content: "Say hi in one short sentence." }],
  maxTokens: 32,
})) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}
```

```python Python theme={null}
from polargrid import PolarGrid

client = await PolarGrid.create(api_key="pg_...")

async for chunk in client.chat_completion_stream({
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Say hi in one short sentence."}],
    "max_tokens": 32,
}):
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)
```

## Capabilities

| Field | Value |
| - | - |
| Architecture | Mixture-of-Experts (256 experts, 8 routed + 1 shared active; \~3B active / 35B total) |
| Context window | 8192 tokens (served; native context is larger, capped here to bound KV-cache VRAM) |
| Streaming | Yes (SSE via `stream: true`) |
| Function calling / tools | Yes (Hermes-style; see "Function calling" below) |
| Structured output (`response_format`) | Yes — `json_object` and `json_schema` (vLLM constrained decoding) |
| Logprobs | No (vllm\_backend exposes only `text_output` over Triton; not surfaced) |
| Sampling controls | `temperature`, `top_p`, `top_k`, `min_p`, `frequency_penalty`, `presence_penalty`, `repetition_penalty`, `seed`, `stop` |
| Reasoning ("thinking") mode | Off by default; opt in via `"enable_thinking": true` in the request body |

## Function calling

Pass OpenAI-shape `tools` and the model returns a `tool_calls` array on the assistant message (or as a `delta.tool_calls` chunk when streaming). The gateway speaks Qwen's Hermes tool-call template under the hood.

```bash cURL theme={null}
curl https://api.<region>.edge.polargrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Whats the weather in Tokyo?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather in a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "tool_choice": "auto"
  }'
```

```typescript JavaScript theme={null}
const reply = await client.chatCompletion({
  model: "qwen-3.6-35b-a3b",
  messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather in a city",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"]
      }
    }
  }],
  tool_choice: "auto",
});
const call = reply.choices[0].message.tool_calls?.[0];
// call.function.name === "get_weather"
// JSON.parse(call.function.arguments) === { city: "Tokyo" }
```

`tool_choice` accepts `"auto"` (model decides), `"none"` (force plain text), `"required"` (force a tool call), or `{ "type": "function", "function": { "name": "<tool>" } }` to force a specific tool.

## Structured output (JSON mode)

Use `response_format` to force the model to emit valid JSON. Backed server-side by vLLM constrained decoding, so the output is guaranteed to parse.

```bash theme={null}
curl https://api.<region>.edge.polargrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Give me a JSON object describing a cat."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age_years": {"type": "integer"},
            "color": {"type": "string"}
          },
          "required": ["name", "age_years", "color"]
        }
      }
    }
  }'
```

`{"type": "json_object"}` accepts any valid JSON; `json_schema` constrains it to your schema.

### Reasoning mode

Qwen3.6 ships with a "thinking" mode that emits a `<think>...</think>` reasoning trace before the user-visible answer. PolarGrid's `/v1/chat/completions` endpoint disables this by default to keep first-token latency low. Opt in per request:

```json theme={null}
{
  "model": "qwen-3.6-35b-a3b",
  "messages": [{"role": "user", "content": "..."}],
  "enable_thinking": true
}
```

## Model identifier

Call this model with the canonical id `qwen-3.6-35b-a3b` at all inference endpoints (`/v1/chat/completions`, `/v1/completions`). The HuggingFace repo id `Qwen/Qwen3.6-35B-A3B-FP8` and the short alias `qwen-3.6-a3b` are accepted at `/v1/models/load` for hot-loading, but inference calls should use the canonical id.

## Aliases

The following caller-facing aliases resolve to `qwen-3.6-35b-a3b`:

| Alias | Resolves to |
| - | - |
| `qwen-3.6-a3b` | `qwen-3.6-35b-a3b` |
| `Qwen/Qwen3.6-35B-A3B-FP8` | `qwen-3.6-35b-a3b` |

## Notes

* **MoE efficiency:** only \~3B of the 35B parameters activate per token, so throughput is closer to a small dense model while quality tracks the full 35B. All expert weights remain resident in VRAM (\~35 GB at FP8).
* **Native FP8** — no runtime quantization step at load.
* **Vision (image input)** is served on the `sfo-02` staging node — OpenAI `image_url` content parts, with context reduced during bring-up. Customer-pilot deployments run the **text-only** path (the vision tower is not loaded). See [Chat Completions → Vision](https://polargrid.mintlify.site/api-reference/chat-completions.md#vision-image-input). `enforce_eager=true` works around a vLLM CUDA-graph path on this model's Gated-DeltaNet hybrid attention, same workaround the retired `qwen-3.5-27b` needed. The current fleet LLM, [`qwen-3.8-27b`](https://polargrid.mintlify.site/models/qwen-3.8-27b), does **not** need it — CUDA-graph capture was re-validated clean on its hybrid attention and it runs `enforce_eager=false`.
* Offered as a customer pilot; co-located with on-edge embeddings for retrieval workloads.

## See also

* [Qwen3.8 27B](https://polargrid.mintlify.site/models/qwen-3.8-27b.md) — sibling dense LLM
* [Authentication](https://polargrid.mintlify.site/authentication.md) — how to mint a JWT from your API key
* [`/v1/models`](https://polargrid.mintlify.site/api-reference/models) — list all available models
* Bench source: [`backend/edge-production-setup/bench/qwen-3.6-35b-a3b/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/qwen-3.6-35b-a3b)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.