Skip to content

Add AI sidecar guide to the SWML guides - #783

Open
Manny-r31 wants to merge 20 commits into
mainfrom
manny/ai-sidecar-guide
Open

Manny-r31 wants to merge 20 commits into
mainfrom
manny/ai-sidecar-guide

Conversation

@Manny-r31

@Manny-r31 Manny-r31 commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Adds Coach live calls with an AI sidecar (/docs/swml/guides/ai-sidecar), a basics guide for the ai_sidecar SWML method. Its structure and phrasing follow the call queues guide in docs(voice): add call queues guide #780.
  • Running example: the document from the ai_sidecar reference page's example: a sales-copilot prompt, hints, and a lookup_account SWAIG function. Webhook URLs and phone numbers are swapped for placeholders. The server answers lookup_account from a stand-in CRM record, so the reader sees a tool call and a tip.
  • Shared sections:
    • The SWML and REST comparison table, prepare, and how the AI sidecar works.
    • Run the sidecar server: a short Python or TypeScript SWMLService that answers the tool at /swaig and prints sidecar callbacks at /events.
  • Coach a call via SWML: the document goes in a hosted SWML Script, whose YAML and JSON the reader pastes into the Dashboard. The reader assigns it to a number and calls between two phones.
  • Coach a call in progress via REST: attach a sidecar by call ID, then stop it while the call continues.
  • Left to the reference pages and Next steps cards: REST ask and poke, tool actions, and options.
  • Reference example: drops answer_on_bridge: true from the ai_sidecar reference example, YAML and JSON. The document answers the call before connect runs, so the setting has no effect (mod_infrastructure/swml.c:4827-4832). The guide's document matches the trimmed example.
  • Nav and index: the page lives in the folder-driven Recipes section of the SWML Guides tab, so swml.yml is unchanged. The fern/llms.txt AI sidecar entry now points at the guide.

Notes for reviewers

  • Hosted script, not served SWML: the document is static, so a hosted SWML Script delivers it. The server only receives callbacks, which avoids an External URL with credentials embedded in it.
  • SDK gap: neither SDK's REST client wraps the calling.ai_sidecar* commands, so the REST section uses the existing EndpointRequestSnippet examples. Per author review, the page doesn't call out the gap.
  • TypeScript SDK quirk the sample works around: registerRoutingCallback redirects unless the callback returns exactly null, so the sample returns null, never undefined. Its /swaig route also neither awaits a handler's promise nor cleanly serializes a FunctionResult, so the tool handler is synchronous and returns .toDict().
  • Facts trace to the platform source (mod_openai sidecar implementation, mod_infrastructure schema), the TypeSpec, and the existing reference pages. Internal detail is kept out: engine, transport, Relay topic names, and billing SKUs.
    • ai_sidecar answers the call itself, so the document has no answer.
    • After calling.ai_sidecar.stop, the call can take a new sidecar.
  • The existing ai_sidecar reference pages disagree with the source in places. The guide avoids these claims, and this PR doesn't fix them; that's a follow-up:
    • They say a sidecar and live_transcribe can't run on one call. The source allows it.
    • They list prompt variables (${global_data.x}, ${local_date}, ${session_uuid}) that the source mostly doesn't support.
    • They say params.transcribe_prompt biases speech recognition. It never reaches the speech engine.
    • permissions.swaig_set_global_data isn't enforced.
    • The settings, back_to_back_functions, extensive_data, and set_meta_data actions do nothing in a sidecar.
    • They say final is always the last callback. After a transfer or hangup action, it arrives before stop.
    • The reference index exposes internal detail: the Relay topic name, curl_code, and the speech engine vendor.
  • Code block titles keep the template's <Language> — <client> format.

Verification

  • yarn fern-md-check: passes (all MDX files valid).
  • yarn fern-check: only the Fern-auth-gated "missing redirects" check fails locally because this machine isn't logged in to Fern. The one warning is the existing accent-color contrast warning.
  • Samples were reviewed statically against the pinned SDK source; none were run. The author is testing them against a live call.
    • Python signalwire-sdk==3.4.1: SWMLService basic auth, define_tool with /swaig dispatch (it reads arguments from argument.parsed, the shape the sidecar sends), and register_routing_callback, which runs on a non-empty POST after the auth check.
    • TypeScript @signalwire/sdk@2.0.5: SWMLService basic auth, defineTool with /swaig dispatch, registerRoutingCallback, and FunctionResult.toDict().
  • Internal links are checked against slug: frontmatter. Anchors are checked by hand: #callback-types, #answer-a-call-via-swml, #prepare-for-your-first-call, and the in-page anchors.

Preview

  • /docs/swml/guides/ai-sidecar
  • /docs/swml/reference/calling/ai-sidecar (example only)

🤖 Generated with Claude Code

New capability guide for ai_sidecar: first-run coaching walkthrough with a
Python and TypeScript SWMLService server, plus REST attach/ask/poke/stop,
prompt and SWAIG tool guidance, callback handling, options, and
troubleshooting. Wires the page into the SWML Guides nav and points the
llms.txt AI sidecar entry at the guide.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

@Manny-r31
Manny-r31 marked this pull request as draft October 6, 2026 17:40
Manny-r31 and others added 12 commits October 6, 2026 13:52
Cut the capability matrix, REST attach/ask/poke/stop sections, tool
actions, options, diagram, JSON twin, and long troubleshooting list.
The guide now covers attaching a sidecar, one tool, reading advice
callbacks, and prompt tips, and links to the reference for the rest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Bring back the template-required product comparison table covering SWML
and REST, and add a short REST section for attaching and stopping a
sidecar on a call in progress.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Follow the layout and phrasing of the call queues guide (#780): shared
comparison table, prepare, how it works, and coaching server sections,
then one section per approach. SWML attaches the sidecar when the call
starts; REST stops it and attaches a new one by call ID.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replace the Bayview Taxi booking-lookup example with a generic support
call: the sidecar sends the agent one short tip per customer turn, with
no tools. The server samples drop the SWAIG tool and /swaig route.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Drop server and troubleshooting asides, add skip to the callback table,
link the SWAIG setting inline, and remove the SDK-gap note.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The document is static, so the SWML approach now pastes it into a hosted
SWML Script instead of pointing an External URL at the server. The server
samples only receive callbacks at /events.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… guide

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Manny-r31
Manny-r31 marked this pull request as ready for review October 7, 2026 15:00
Manny-r31 and others added 5 commits October 7, 2026 11:22
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Use the ai_sidecar reference example (sales copilot prompt, hints, and a
lookup_account SWAIG function) as the guide's SWML document, and add a
/swaig handler for lookup_account to the server samples.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The document answers the call before connect runs, so answer_on_bridge
has no effect. Remove it from the guide and from the ai_sidecar
reference example it's based on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hey-august hey-august mentioned this pull request Oct 22, 2026
1 of 13 tasks
Manny-r31 and others added 2 commits October 26, 2026 10:11
…k and poke

The guide now lives at /docs/platform/ai/sidecar as a general guide for
the AI sidecar, with task sections for attaching, asking a question,
prompting a tip, and stopping a sidecar via SWML and the REST Calling API.
Adds a card on the AI overview and an llms.txt entry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants