Skip to content

feat: scripts pause for approvals, clearer approval cards, sim hand-over - #90

Merged
albanm merged 19 commits into
mainfrom
chore-simulation-approval
Nov 9, 2026
Merged

albanm merged 19 commits into
mainfrom
chore-simulation-approval

Conversation

@albanm

@albanm albanm commented Nov 9, 2026

Copy link
Copy Markdown
Member

Fixes what the portals autonomous-agent simulations exposed around approvals: 19 cards, 2 approved, 15 cancelled by the simulated person typing, 0 refused; 7 of 25 reviews timed out at stage 1.

What changed

  • Simulation runner (lib-sim): a turn stopped on an approval card is handed to the person like a wait; the persona is told a card is on screen; a decision that ends on a new card holds the persona's message back. Cards carry data-approval-id for that.
  • Reviewer timeout: approvalReview.stage1TimeoutMs (APPROVAL_REVIEW_STAGE1_TIMEOUT_MS), 5 s by default, 15 s in development (the dev bridge's stage 1 ran p90 4.2 s, 20 % past 5 s).
  • Cancelled approvals: a card cancelled by a new message or a page action reads « Annulé », not « Refusé »; the card says « Envoyer un message annule cette demande. »
  • Reviewer reason for the person: stage 2 also writes personReason in the people's language, shown on cards and reviewer-denial chips; the model keeps reading the English reason.
  • Scripts pause for a person: a code_mode nested call routed to a person (or judged unsure in a chat) pauses the script through @ai-sdk/code-mode 1.0.116's signed continuations, shows the nested call's card, and resumes after the decision without re-running earlier calls. Continuations live in a new script-pauses collection (TTL 24 h, dropped with the thread). The signing key is derived from the cipher key (HKDF over CIPHER_PASSWORD's key): no new secret to deploy. Exposure and usage limits carry across a pause; each decision of parallel pending calls is kept.
  • Reviewer judgement: 11 guard-eval cases from the portals runs (3 adversarial); a display-option reading for stage 2 only and a stage-1 clause flagging ambiguous targets. Full corpus on the dev bridge (Haiku, ×3): 219/225 as expected, no harmful call allowed, stage 2 on 79 % of stage-1 reviews (64 % before).
  • Docs: code-mode, autonomous-agents, moderation, configuration, lib-sim README.

Why: in the simulations the approval flow, not the agent, decided most failures: the runner hid cards from the persona, reviewer timeouts turned writes into cards, cards were unreadable for a French non-technical person, and scripts could not ask a person.

Regression risks

  • Code mode in a chat: tools that need a person are now callable from scripts (codeCallable with person), and an unsure nested call pauses instead of throwing. A task run keeps the old behaviour. code_mode's description changes for the model.
  • Rejection inside a script: continueCodeModeApproval(approved: false) fails the whole resume in 1.0.116, so a rejection is resumed as an approval and our wrapper throws rejectedCallMessage for that call. Pinned by unit tests, including parallel pending calls; relies on the package's replay semantics (pin @ai-sdk/code-mode at 1.0.116, it is experimental_).
  • Resume can fail closed: a continuation whose tool set changed (page tools on the deciding tab, an MCP listing change) or that expired is refused before anything runs; logged as code-mode-resume-refused.
  • Changing CIPHER_PASSWORD now also invalidates scripts paused at that moment (on top of the stored provider keys).
  • Reviewer prompt changed (stage-1 instruction, stage-2 reading, personReason in the verdict schema): measured on Haiku via the dev bridge only, not on the production moderator models. Stage 2 runs more often (+1 s median, +5 % tokens).
  • New required config key approvalReview.stage1TimeoutMs (defaulted in default.js): a deployment overriding the whole config object must keep it.
  • New script-pauses collection, created with its indexes by mongo.init: no upgrade script.

albanm and others added 19 commits November 6, 2026 15:48
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… bridge

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…lls in the latency cases

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ing cancels it

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… the card

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…the chip

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…he decision

In a chat, a script's call that needs the person (reviewer unsure, an allow
on the reviewer+person route, a personal assistant's write) pauses the
script through @ai-sdk/code-mode's interrupt mode. The signed continuation
is kept in a script-pauses collection (24 h TTL), the code_mode call becomes
the card of the nested call, and the approval run resumes the script where
it stopped before its first step. A rejection throws inside the script; a
loop asks again after each decision; a new message cancels it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s; pauses go with the thread

Calls pending together in one continuation run only after the last decision:
every decision is now stored with the continuation, so a rejected parallel call
throws instead of running and an approved one is not reviewed again. A paused
script's continuation is dropped when its card is cancelled and when its
conversation is archived, purged, erased, or its agent deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… people meant them

Five reviewer cases from the portals runs in the guard eval, the rb-portal-*
fixtures rebuilt as one new box on the home page, and a rule: a display option
of an item the agent added for the request is part of that request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its scope

The rule now excludes what an item shows, embeds or links to and who can see
or edit it, and never lifts the other rules; three boundary cases pin it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1 flags ambiguous targets

In the shared policy, stage 1's one-word screen read the display-option rule as
a clearance and cleared harmful calls. It moves to stage 2's restated task, for
actions, only for a path that plainly names the added item; stage 1 flags a call
whose index, path or id could name another item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rs attributed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… secret is optional

Calls made before a pause are answered from run's record, so the resuming turn is
handed their count, the bytes they moved and the tools they read (stored with the
pause). Without CODE_MODE_CONTINUATION_SECRET no script pauses and the api logs
code-mode-continuation-secret-missing. A refused resume logs the package's message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p is cancelled

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ona's message back

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… scripts, optional secret

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scripts pause in a chat with a single mode again: codeMode.continuationSecret is required
(at least 32 characters, the same on every instance), the startup error and the no-secret
branches are gone. The usage carried across a pause and the resume-refused log stay.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… cipher key

No CODE_MODE_CONTINUATION_SECRET to deploy: the key is HKDF over the cipher key, which
CIPHER_PASSWORD already makes the same on every instance and across restarts, so a
decision resumes a paused script wherever it lands. The package's default random key
per process would fail every pending card at a restart or on another instance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@albanm
albanm merged commit 1c2850b into main Nov 9, 2026
4 checks passed
@albanm
albanm deleted the chore-simulation-approval branch November 9, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant