Skip to content

fix: inject readImage result once and strip base64 from tool message - #1306

Merged
avoidwork merged 4 commits into
mainfrom
fix/image-dispatch-inject-once
Oct 6, 2026
Merged

avoidwork merged 4 commits into
mainfrom
fix/image-dispatch-inject-once

Conversation

@avoidwork

@avoidwork avoidwork commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Description

Implements the image-dispatch inject-once fix. The ImageDispatch middleware now injects a readImage result only once — tracked by tool_call_id in a closure-level Set — so the image is attached on the turn immediately following the call and never re-attached to unrelated subsequent prompts. It pairs the image with the prompt that triggered the readImage call (the HumanMessage preceding the AIMessage that made the call) rather than the most recent user prompt, and strips the base64 payload from the readImage ToolMessage content by replacing it with a stub ToolMessage ("Image read successfully.") so the model never receives it as plain text tokens. The replacement message list is built without mutating the persisted state objects that request.messages shares with the checkpointer.

Type of Change

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor (no functional changes)
  • Performance improvement
  • CI / build / tooling

Testing

Added regression tests in tests/unit/provider/imageDispatchMiddleware.test.js covering: base64 payload stripping from the ToolMessage content, pairing the image with the triggering prompt (not the most recent one), and inject-once behavior (no re-attachment to a later unrelated prompt). All 12 tests in tests/unit/provider/imageDispatchMiddleware.test.js pass. npm run lint passes with 0 warnings and 0 errors. npm run coverage reports imageDispatchMiddleware.js at 96.00% line coverage (84.21% functions, 83.33% branches), consistent with the pre-change baseline of 96.26%. Note: the full npm run coverage run reports a pre-existing, unrelated failure in tests/unit/tools/web.test.js (web tool default search engine config) that also fails on main and is not touched by this change.

Coverage

  • Line coverage maintained

Checklist

  • npm run lint passes
  • Tests pass with maintained line coverage
  • No forbidden patterns used
  • Conventional Commit style applied

Closes #1305

@avoidwork avoidwork self-assigned this Oct 6, 2026
@avoidwork avoidwork changed the title docs: add OpenSpec change for image-dispatch inject-once fix: inject readImage result once and strip base64 from tool message Oct 6, 2026
@avoidwork

Copy link
Copy Markdown
Owner Author

Audit Results

Goal Fulfillment

All three goals from issue #1305 are satisfied:

  1. Inject once, not every turn — The middleware now tracks dispatched readImage tool_call_ids in a closure-level Set. The image is injected only on the turn immediately following the readImage call and is not re-attached to unrelated subsequent prompts.
  2. Pair with the triggering prompt — The middleware records the HumanMessage that preceded the AIMessage making the readImage call (via a Map keyed on tool_call_id) and uses that text for the injected content block, not the most recent user prompt.
  3. Stop sending base64 as text — The middleware replaces the readImage ToolMessage content with a stub ToolMessage ("Image read successfully.") in the request message list, so the base64 payload never reaches the model as plain text tokens. It does this by building a replacement message list rather than mutating persisted state objects shared with the checkpointer.

Spec Compliance

The implementation matches the delta spec for read-image-vision-dispatch:

  • readImage ToolMessage triggers dispatch — ✅
  • Image is injected only once — ✅
  • Image is paired with the triggering prompt — ✅
  • Base64 payload is stripped from the ToolMessage — ✅
  • ok:false result is skipped — ✅
  • Multiple readImage results in one turn — ✅
  • No readImage messages is a no-op — ✅

Task Completion

All 8 tasks in tasks.md are marked complete.

Quality Check

  • All 12 imageDispatchMiddleware.test.js tests pass (including 3 new regression tests).
  • npm run lint passes (0 warnings, 0 errors).
  • imageDispatchMiddleware.js at 96% line coverage.
  • The full npm run coverage reports a pre-existing, unrelated failure in tests/unit/tools/web.test.js (web tool default search engine config, caused by a TAVILY_API_KEY env var in the environment) that also fails on main — not introduced by this change.

@avoidwork
avoidwork merged commit b243be3 into main Oct 6, 2026
2 checks passed
@avoidwork
avoidwork deleted the fix/image-dispatch-inject-once branch October 6, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: image-dispatch re-injects base64 and pairs image with wrong prompt

1 participant