> ## Documentation Index > Fetch the complete documentation index at: https://polargrid.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Qwen3.6 35B-A3B > Qwen3.6 35B-A3B MoE text LLM on vLLM with native FP8 weights Qwen3.6 35B-A3B (`qwen-3.6-35b-a3b`) is a Mixture-of-Experts text LLM served on PolarGrid edge nodes via Triton's `vllm_backend`. It has **35 billion total parameters** but activates only **\~3 billion per token** (256 experts, 8 routed + 1 shared active). Weights ship pre-quantized to FP8 (\~35 GB VRAM) and load directly on PolarGrid's Blackwell edge GPUs without runtime requantization. * **HF repo:** [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8) * **Modality:** Text LLM (MoE) * **Backend:** Triton `vllm` (LLM pod) - **Available regions:** `yul-02` (customer pilot — limited availability) ## Headline benchmark We publish **two** numbers side by side. End-to-end is what your application actually experiences (request → response, network included). Server-only is what the GPU spends on inference (apples-to-apples vs centralized providers' published "inference-only" figures). The gap is the latency PolarGrid's `yul-02` PoP eliminates by being at the edge. | Measurement | TTFT p50 | TTFT p95 | Throughput p50 | | --------------------------------- | ---------- | -------- | -------------- | | **End-to-end (with network)** | **212 ms** | 238 ms | **15.8 tok/s** | | **Server-only (no network)** | **163 ms** | 174 ms | — | | *Network overhead (e2e − server)* | *49 ms* | — | — | *Bench: 100 streaming chat-completion runs against `https://api.yul-02.edge.polargrid.ai`, captured 2026-06-09 from yvr-01 (Vancouver) over the public internet. End-to-end is client wall-clock; server-only is read from the gateway's `pg_metadata` SSE event (`inference_ttft_ms` / `inference_total_ms`). Raw runs: [`benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json`](https://github.com/PolarGrid-AI/polargrid-monorepo/blob/main/benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json).* > **Apples-to-apples disclaimer.** Other providers usually publish only their server-side number; comparing it to our **server-only** row is the fair baseline. Our **end-to-end** row is what you'll see from a customer-side request because PolarGrid runs at the edge — the network row above shows exactly how much that's worth in milliseconds. ## Quickstart ```bash cURL theme={null} curl https://api.yul-02.edge.polargrid.ai/v1/chat/completions \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "qwen-3.6-35b-a3b", "messages": [{"role": "user", "content": "Say hi in one short sentence."}], "stream": true, "max_tokens": 32 }' ``` ```typescript JavaScript theme={null} import { PolarGrid } from "@polargrid/polargrid-sdk"; const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY }); for await (const chunk of client.chatCompletionStream({ model: "qwen-3.6-35b-a3b", messages: [{ role: "user", content: "Say hi in one short sentence." }], maxTokens: 32, })) { const content = chunk.choices[0]?.delta?.content; if (content) process.stdout.write(content); } ``` ```python Python theme={null} from polargrid import PolarGrid client = await PolarGrid.create(api_key="pg_...") async for chunk in client.chat_completion_stream({ "model": "qwen-3.6-35b-a3b", "messages": [{"role": "user", "content": "Say hi in one short sentence."}], "max_tokens": 32, }): content = chunk.choices[0].delta.content if content: print(content, end="", flush=True) ``` ## Capabilities | Field | Value | | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | Architecture | Mixture-of-Experts (256 experts, 8 routed + 1 shared active; \~3B active / 35B total) | | Context window | 8192 tokens (served; native context is larger, capped here to bound KV-cache VRAM) | | Streaming | Yes (SSE via `stream: true`) | | Function calling / tools | Yes (Hermes-style; see "Function calling" below) | | Structured output (`response_format`) | Yes — `json_object` and `json_schema` (vLLM constrained decoding) | | Logprobs | No (vllm\_backend exposes only `text_output` over Triton; not surfaced) | | Sampling controls | `temperature`, `top_p`, `top_k`, `min_p`, `frequency_penalty`, `presence_penalty`, `repetition_penalty`, `seed`, `stop` | | Reasoning ("thinking") mode | Off by default; opt in via `"enable_thinking": true` in the request body | ## Function calling Pass OpenAI-shape `tools` and the model returns a `tool_calls` array on the assistant message (or as a `delta.tool_calls` chunk when streaming). The gateway speaks Qwen's Hermes tool-call template under the hood. ```bash cURL theme={null} curl https://api.yul-02.edge.polargrid.ai/v1/chat/completions \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "qwen-3.6-35b-a3b", "messages": [{"role": "user", "content": "Whats the weather in Tokyo?"}], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather in a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"] } } }], "tool_choice": "auto" }' ``` ```typescript JavaScript theme={null} const reply = await client.chatCompletion({ model: "qwen-3.6-35b-a3b", messages: [{ role: "user", content: "What's the weather in Tokyo?" }], tools: [{ type: "function", function: { name: "get_weather", description: "Get the current weather in a city", parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"] } } }], tool_choice: "auto", }); const call = reply.choices[0].message.tool_calls?.[0]; // call.function.name === "get_weather" // JSON.parse(call.function.arguments) === { city: "Tokyo" } ``` `tool_choice` accepts `"auto"` (model decides), `"none"` (force plain text), `"required"` (force a tool call), or `{ "type": "function", "function": { "name": "" } }` to force a specific tool. ## Structured output (JSON mode) Use `response_format` to force the model to emit valid JSON. Backed server-side by vLLM constrained decoding, so the output is guaranteed to parse. ```bash theme={null} curl https://api.yul-02.edge.polargrid.ai/v1/chat/completions \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "qwen-3.6-35b-a3b", "messages": [{"role": "user", "content": "Give me a JSON object describing a cat."}], "response_format": { "type": "json_schema", "json_schema": { "schema": { "type": "object", "properties": { "name": {"type": "string"}, "age_years": {"type": "integer"}, "color": {"type": "string"} }, "required": ["name", "age_years", "color"] } } } }' ``` `{"type": "json_object"}` accepts any valid JSON; `json_schema` constrains it to your schema. ### Reasoning mode Qwen3.6 ships with a "thinking" mode that emits a `...` reasoning trace before the user-visible answer. PolarGrid's `/v1/chat/completions` endpoint disables this by default to keep first-token latency low. Opt in per request: ```json theme={null} { "model": "qwen-3.6-35b-a3b", "messages": [{"role": "user", "content": "..."}], "enable_thinking": true } ``` ## Model identifier Call this model with the canonical id `qwen-3.6-35b-a3b` at all inference endpoints (`/v1/chat/completions`, `/v1/completions`). The HuggingFace repo id `Qwen/Qwen3.6-35B-A3B-FP8` and the short alias `qwen-3.6-a3b` are accepted at `/v1/models/load` for hot-loading, but inference calls should use the canonical id. ## Aliases The following caller-facing aliases resolve to `qwen-3.6-35b-a3b`: | Alias | Resolves to | | -------------------------- | ------------------ | | `qwen-3.6-a3b` | `qwen-3.6-35b-a3b` | | `Qwen/Qwen3.6-35B-A3B-FP8` | `qwen-3.6-35b-a3b` | ## Notes * **MoE efficiency:** only \~3B of the 35B parameters activate per token, so throughput is closer to a small dense model while quality tracks the full 35B. All expert weights remain resident in VRAM (\~35 GB at FP8). * **Native FP8** — no runtime quantization step at load. * Runs the **text-only** path (vision encoder not served); `enforce_eager=true` works around a vLLM CUDA-graph path on this model's Gated-DeltaNet hybrid attention, same workaround as `qwen-3.5-27b`. * Customer pilot on `yul-02` (Montreal). Co-located with on-edge embeddings for retrieval workloads. ## See also * [Qwen3.5 27B](/models/qwen-3.5-27b) — sibling dense LLM * [Authentication](/authentication) — how to mint a JWT from your API key * [`/v1/models`](/api-reference/models) — list all available models * Bench source: [`backend/edge-production-setup/bench/qwen-3.6-35b-a3b/`](https://github.com/PolarGrid-AI/polargrid-monorepo/tree/main/backend/edge-production-setup/bench/qwen-3.6-35b-a3b)