Qwen3.6 35B-A3B

Qwen3.6 35B-A3B MoE text LLM on vLLM with native FP8 weights

Qwen3.6 35B-A3B (qwen-3.6-35b-a3b) is a Mixture-of-Experts text LLM served on PolarGrid edge nodes via Triton’s vllm_backend. It has 35 billion total parameters but activates only ~3 billion per token (256 experts, 8 routed + 1 shared active). Weights ship pre-quantized to FP8 (~35 GB VRAM) and load directly on PolarGrid’s Blackwell edge GPUs without runtime requantization.

• HF repo: Qwen/Qwen3.6-35B-A3B-FP8
• Modality: Text LLM (MoE)
• Backend: Triton vllm (LLM pod)

• Available regions: yul-02 (customer pilot — limited availability)

Headline benchmark

We publish two numbers side by side. End-to-end is what your application actually experiences (request → response, network included). Server-only is what the GPU spends on inference (apples-to-apples vs centralized providers’ published “inference-only” figures). The gap is the latency PolarGrid’s yul-02 PoP eliminates by being at the edge.

Measurement	TTFT p50	TTFT p95	Throughput p50
End-to-end (with network)	212 ms	238 ms	15.8 tok/s
Server-only (no network)	163 ms	174 ms	—
Network overhead (e2e − server)	49 ms	—	—

Bench: 100 streaming chat-completion runs against https://api.yul-02.edge.polargrid.ai, captured 2026-06-09 from yvr-01 (Vancouver) over the public internet. End-to-end is client wall-clock; server-only is read from the gateway’s pg_metadata SSE event (inference_ttft_ms / inference_total_ms). Raw runs: benchmarks/yul-02-2026-06-09/35b-a3b/llm_bench.json.

Apples-to-apples disclaimer. Other providers usually publish only their server-side number; comparing it to our server-only row is the fair baseline. Our end-to-end row is what you’ll see from a customer-side request because PolarGrid runs at the edge — the network row above shows exactly how much that’s worth in milliseconds.

Quickstart

cURL

curl https://api.yul-02.edge.polargrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Say hi in one short sentence."}],
    "stream": true,
    "max_tokens": 32
  }'

import { PolarGrid } from "@polargrid/polargrid-sdk";

const client = await PolarGrid.create({ apiKey: process.env.POLARGRID_API_KEY });

for await (const chunk of client.chatCompletionStream({
  model: "qwen-3.6-35b-a3b",
  messages: [{ role: "user", content: "Say hi in one short sentence." }],
  maxTokens: 32,
})) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}

from polargrid import PolarGrid

client = await PolarGrid.create(api_key="pg_...")

async for chunk in client.chat_completion_stream({
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Say hi in one short sentence."}],
    "max_tokens": 32,
}):
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Capabilities

Field	Value
Architecture	Mixture-of-Experts (256 experts, 8 routed + 1 shared active; ~3B active / 35B total)
Context window	8192 tokens (served; native context is larger, capped here to bound KV-cache VRAM)
Streaming	Yes (SSE via stream: true)
Function calling / tools	Yes (Hermes-style; see “Function calling” below)
Structured output (response_format)	Yes — json_object and json_schema (vLLM constrained decoding)
Logprobs	No (vllm_backend exposes only text_output over Triton; not surfaced)
Sampling controls	temperature, top_p, top_k, min_p, frequency_penalty, presence_penalty, repetition_penalty, seed, stop
Reasoning (“thinking”) mode	Off by default; opt in via "enable_thinking": true in the request body

Function calling

Pass OpenAI-shape tools and the model returns a tool_calls array on the assistant message (or as a delta.tool_calls chunk when streaming). The gateway speaks Qwen’s Hermes tool-call template under the hood.

cURL

curl https://api.yul-02.edge.polargrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Whats the weather in Tokyo?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather in a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "tool_choice": "auto"
  }'

const reply = await client.chatCompletion({
  model: "qwen-3.6-35b-a3b",
  messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather in a city",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"]
      }
    }
  }],
  tool_choice: "auto",
});
const call = reply.choices[0].message.tool_calls?.[0];
// call.function.name === "get_weather"
// JSON.parse(call.function.arguments) === { city: "Tokyo" }

tool_choice accepts "auto" (model decides), "none" (force plain text), "required" (force a tool call), or { "type": "function", "function": { "name": "<tool>" } } to force a specific tool.

Structured output (JSON mode)

Use response_format to force the model to emit valid JSON. Backed server-side by vLLM constrained decoding, so the output is guaranteed to parse.

curl https://api.yul-02.edge.polargrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.6-35b-a3b",
    "messages": [{"role": "user", "content": "Give me a JSON object describing a cat."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age_years": {"type": "integer"},
            "color": {"type": "string"}
          },
          "required": ["name", "age_years", "color"]
        }
      }
    }
  }'

{"type": "json_object"} accepts any valid JSON; json_schema constrains it to your schema.

Reasoning mode

Qwen3.6 ships with a “thinking” mode that emits a <think>...</think> reasoning trace before the user-visible answer. PolarGrid’s /v1/chat/completions endpoint disables this by default to keep first-token latency low. Opt in per request:

{
  "model": "qwen-3.6-35b-a3b",
  "messages": [{"role": "user", "content": "..."}],
  "enable_thinking": true
}

Model identifier

Call this model with the canonical id qwen-3.6-35b-a3b at all inference endpoints (/v1/chat/completions, /v1/completions). The HuggingFace repo id Qwen/Qwen3.6-35B-A3B-FP8 and the short alias qwen-3.6-a3b are accepted at /v1/models/load for hot-loading, but inference calls should use the canonical id.

Aliases

The following caller-facing aliases resolve to qwen-3.6-35b-a3b:

Alias	Resolves to
qwen-3.6-a3b	qwen-3.6-35b-a3b
Qwen/Qwen3.6-35B-A3B-FP8	qwen-3.6-35b-a3b

Notes

• MoE efficiency: only ~3B of the 35B parameters activate per token, so throughput is closer to a small dense model while quality tracks the full 35B. All expert weights remain resident in VRAM (~35 GB at FP8).
• Native FP8 — no runtime quantization step at load.
• Runs the text-only path (vision encoder not served); enforce_eager=true works around a vLLM CUDA-graph path on this model’s Gated-DeltaNet hybrid attention, same workaround as qwen-3.5-27b.
• Customer pilot on yul-02 (Montreal). Co-located with on-edge embeddings for retrieval workloads.

See also

• Qwen3.5 27B — sibling dense LLM
• Authentication — how to mint a JWT from your API key
• /v1/models — list all available models
• Bench source: backend/edge-production-setup/bench/qwen-3.6-35b-a3b/