Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions fern/composer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ Composer understands voice agent architecture and Vapi's capabilities. It can cr
- **Cannot make business logic decisions**: You provide the domain knowledge; Composer handles the technical configuration.
- **Cannot test every behavior**: Suites Composer generates run in chat, so they can't check voice-only behavior such as pronunciation, and [some handoffs and transfers](/observability/simulations-advanced#handoffs-and-transfers-in-simulations) aren't tested. Make test calls yourself for these.
- **Cannot deploy to production automatically**: You control when and how changes go live.
- **Daily limits on test generation**: Requirement drafts, suite generations, and suite edits share a free daily allowance per organization with the [Simulations onboarding API](/observability/simulations-generate-api#limits). When your organization reaches a limit, Composer tells you and doesn't retry. The limits reset at 00:00 UTC.

## Common use cases

Expand Down
3 changes: 3 additions & 0 deletions fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -692,6 +692,9 @@ navigation:
- page: Advanced
path: observability/simulations-advanced.mdx
icon: fa-light fa-flask-vial
- page: Generate with the API
path: observability/simulations-generate-api.mdx
icon: fa-light fa-wand-magic-sparkles
- page: Configure an AI tester
path: observability/simulations-configure-ai-tester.mdx
icon: fa-light fa-robot
Expand Down
2 changes: 2 additions & 0 deletions fern/observability/simulations-advanced.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,8 @@ For a transient scenario, set the same `targetOverrides` on the inline `scenario
| --- | --- |
| Every iteration fails | Review the transcript, then verify the evaluation's description, data type, comparator, and expected value. |
| A run stays queued or running | Check the assistant or squad configuration and its provider credentials. Fetch the run and its items to find a failure reason. |
| Creating a voice run returns `400 Target assistant would fail to start a call: <message>` | The inline assistant in `target` has a configuration a call would reject, such as a tool type the model doesn't support. Fix what the message names, then create the run again. Vapi checks this only for voice runs with an inline assistant target, not for a saved `assistantId`, a squad, or a chat run. |
| A run item fails with `Failed to create target call: <message>` | The assistant or squad under test failed call validation when the run item started its call. Fix what the message names, then rerun. No AI tester call is started for that item. |
| Results vary between runs | Make the intent and evaluations more specific, then run multiple iterations to measure consistency. |
| There is no audio or recording | Check whether the run used chat mode. Use voice mode to test or record audio. |
| Start or end webhooks do not trigger | Confirm the corresponding hook and its URL are configured on the scenario. |
Expand Down
Loading
Loading