Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ai/rocky-ai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Set an intent and enable [automatic repair](../resolve/automatic-check-repair) f

## Consumption and Invocations

Every root cause analysis — whether triggered automatically or manually — counts as one invocation against your plan's Resolve quota. If the same analysis result is pushed to multiple alert channels, it is still counted as a single invocation. An automatic check repair counts as one additional invocation. AI Assistant conversations and **Initialize from saved check** do not currently count against this allowance.
Every root cause analysis — whether triggered automatically or manually — counts as one invocation against your plan's Resolve quota. If the same analysis result is pushed to multiple alert channels, it is still counted as a single invocation. An automatic check repair counts as one additional invocation. Each AI Assistant conversation counts as one invocation when it first runs. **Initialize from saved check** does not currently count against this allowance.

When your invocation quota is reached, no overages are charged. Invocations refresh at the start of your next billing cycle. You can monitor your current usage and configure automatic analysis in the [AI settings](https://app.checklyhq.com/accounts/rocky-ai).

Expand Down
44 changes: 35 additions & 9 deletions detect/synthetic-monitoring/agentic-checks/migrate.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,50 @@ hidden: true
Agentic Checks are deprecated and will stop running on **December 1, 2026**. Replace each active Agentic Check before that date to keep monitoring its journey.
</Warning>

Create a Browser or Multistep Check for the same journey, verify the replacement, then deactivate the Agentic Check. The AI Assistant can create other check and monitor types too; this guide uses Browser and Multistep Checks because automatic repair can edit their scripts in Checkly. The replacement has its own results and history.
Create a Browser or Multistep Check for the same journey, verify the replacement, then deactivate the Agentic Check. This guide uses Browser and Multistep Checks because automatic repair can edit their scripts in Checkly. The replacement has its own results and history.

How you create the replacement depends on your plan:

- **Trial, Starter, Team, and Enterprise:** use the [AI Assistant](../../../ai/ai-assistant) in Checkly.
- **Hobby:** use your own AI agent connected to the [Checkly MCP Server](../../../ai/mcp-server).

<Accordion title="Prerequisites">

- A Trial, Starter, Team, or Enterprise account with access to the [AI Assistant](../../../ai/ai-assistant)
- Permission to create and edit checks
- Access to each Agentic Check's prompt, schedule, locations, alert settings, and any variables or secrets it uses
- For the AI Assistant: Rocky AI enabled for the account. An account owner can enable it in the [AI settings](https://app.checklyhq.com/accounts/rocky-ai).
- For your own agent: an MCP client connected to the Checkly MCP Server. See [Set up the MCP server](../../../ai/mcp-server/setup).

</Accordion>

## Replace an Agentic Check
<Note>
If a Checkly CLI project manages the Agentic Check, create the replacement in that project instead: replace the `AgenticCheck` construct with a `BrowserCheck` or `MultiStepCheck` construct and deploy. [Checkly Skills](../../../ai/skills) help your agent write and test the check. Changes made in the Checkly app or through the MCP Server to a check that a CLI project manages are overwritten by its next deploy.
</Note>

## Create the replacement with the AI Assistant

1. Open the Agentic Check and copy its prompt. Note its frequency, locations, alert channels, referenced variables or secrets, and any custom skills.
2. Open **AI Assistant**, select **New**, and ask it to create a check for the same journey. Give it the original prompt and the pass and fail criteria. Choose a **Browser Check** for a web journey or a **Multistep Check** for an API workflow.
1. Open the Agentic Check and select **Copy prompt**. Note its frequency, locations, alert channels, referenced variables or secrets, and any custom skills.
2. Select **Open AI assistant**, paste the prompt, and ask for a check for the same journey. Include the pass and fail criteria. Choose a **Browser Check** for a web journey or a **Multistep Check** for an API workflow.
3. Review the proposed plan and select **Create checks**. Inspect the generated script, assertions, settings, and check intent. Confirm that important outcomes and guardrails from the original prompt are present. If the Agentic Check used custom skills, verify that the replacement script covers the behavior you still need.
4. Run the replacement check and confirm it reports the expected result. Correct the script or settings if needed. Match the original check's frequency, locations, and alerting before relying on the replacement.
5. In the replacement check's **Automatic Repair** settings, review its intent, select **On** under **Automatic repair behavior**, and save. [Learn how automatic repair works](../../../resolve/automatic-check-repair).
6. Once the replacement is running as expected, deactivate the Agentic Check. Keep it for its existing configuration and history, or remove it later according to your retention needs.

## Create the replacement with your own AI agent

1. Open the Agentic Check and select **Copy prompt**. Note its frequency, locations, alert channels, referenced variables or secrets, and any custom skills.
2. In your MCP client, ask your agent to create the replacement through the Checkly MCP Server. Give it the copied prompt, the check type, and the original schedule and locations:

```text title="Prompt" wrap
Use Checkly to create a Browser Check named "Checkout flow" that monitors the journey below. Run it every 10 minutes from eu-west-1 and us-east-1. The prompt references variables as {{VARIABLE_NAME}}; read them in the script with process.env.VARIABLE_NAME. Show me the script before you create the check.

<paste the Agentic Check prompt here>
```

3. Open the new check in Checkly and inspect its script, assertions, and settings. Confirm that important outcomes and guardrails from the original prompt are present. If the Agentic Check used custom skills, verify that the replacement script covers the behavior you still need.

## Verify and switch over

1. Run the replacement check and confirm it reports the expected result. Correct the script or settings if needed. Match the original check's frequency, locations, and alerting before relying on the replacement.
2. In the replacement check's **Automatic Repair** settings, review its intent, select **On** under **Automatic repair behavior**, and save. If the check has no intent yet, select **Initialize from saved check** first. [Learn how automatic repair works](../../../resolve/automatic-check-repair).
3. Once the replacement is running as expected, deactivate the Agentic Check. Keep it for its existing configuration and history, or remove it later according to your retention needs.

<Tip>
Migrate one journey at a time. If the original prompt covered several unrelated flows, create a separate replacement check for each flow and verify each one before deactivating the Agentic Check.
Expand All @@ -39,4 +65,4 @@ You can enable automatic repair on Browser and Multistep Checks you already use.

## AI invocation usage

AI Assistant conversations and **Initialize from saved check** currently do not consume your Resolve AI invocation allowance. Root cause analysis and a subsequent automatic repair each consume an invocation. See [AI invocation usage](../../../resolve/automatic-check-repair#ai-invocation-usage).
Each AI Assistant conversation uses one AI invocation from your plan's allowance when it first runs. Root cause analysis and a subsequent automatic repair each use an invocation. **Initialize from saved check** does not currently use one, and neither does creating checks through the MCP Server. See [AI invocation usage](../../../resolve/automatic-check-repair#ai-invocation-usage).
2 changes: 1 addition & 1 deletion resolve/automatic-check-repair.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ You can also write the intent yourself. Use this option when the script does not

## AI invocation usage

Each new root cause analysis uses one AI invocation. If Checkly also starts a repair, that repair uses one additional invocation from the same Resolve allowance. Each click on **Initialize from saved check** requests a new AI suggestion, but it does not currently draw from that allowance. AI Assistant conversations do not draw from it either. See [Rocky AI consumption and invocations](../ai/rocky-ai#consumption-and-invocations).
Each new root cause analysis uses one AI invocation. If Checkly also starts a repair, that repair uses one additional invocation from the same Resolve allowance. Each click on **Initialize from saved check** requests a new AI suggestion, but it does not currently draw from that allowance. Each AI Assistant conversation uses one invocation from the same allowance when it first runs. See [Rocky AI consumption and invocations](../ai/rocky-ai#consumption-and-invocations).
Loading