The Modular Pipeline Agent orchestrates three models from the PolarGrid catalog on each conversation turn: speech-to-text, an LLM, and text-to-speech. Audio streams in both directions. A parallel event channel carries JSON messages (transcripts, per-token LLM output, latency markers, errors). Any model returned by GET /v1/models with the matching modality can be used.
The agent runs on-device voice activity detection (VAD) to detect end-of-speech, transcribes the utterance, then streams LLM generation into TTS synthesis: TTS begins at the first sentence boundary of LLM output rather than after the full response. If the user starts speaking again while the agent is mid-response, the in-flight LLM and TTS are cancelled.
The server emits the following JSON events on the event channel. Some event types are emitted more than once per turn with different payloads — for example, tts_complete fires once when the first audio plays and again when synthesis finishes.
Function calling (tool use) is supported at the chat completions API level — the gateway’s /v1/chat/completions endpoint supports tools and tool_choice parameters with models that have native tool-use support (qwen-3.8-27b).The voice agent pipeline does not yet integrate function calling. Tool use in the voice agent is planned but not implemented — the pipeline currently streams text tokens only, with no tool-call detection, TTS pausing, or tool-result injection.PersonaPlex does not support function calling (single audio-native model, no LLM step).
PersonaPlex uses a single multi-modal model and exposes a persona prompt and fixed voice IDs. The Modular Pipeline Agent runs three models you pick independently and exposes a per-turn event stream with latency markers. See the comparison table on the PersonaPlex page.
