Skip to content

fix: parse separate JSON objects in skill selection replies - #641

Merged
davidmckayv merged 5 commits into
CopilotKit:mainfrom
jiangjiang248:fix/selection-json-extraction
Sep 26, 2026
Merged

davidmckayv merged 5 commits into
CopilotKit:mainfrom
jiangjiang248:fix/selection-json-extraction

Conversation

@jiangjiang248

Copy link
Copy Markdown
Contributor

Summary

Parse complete JSON objects individually in skill-selection replies. A valid selection now survives surrounding prose and an additional JSON object, while malformed replies retain the existing fallback behavior.

Verification

  • Reproduced the regression on the original implementation.
  • 39 related unit and integration tests passed.
  • Repository lint, typecheck, and format checks passed.
  • The full test suite could not complete in this environment because it requires a dedicated PostgreSQL test database and other local services.

Prepared with AI assistance; I reviewed the diff and test results.

@davidmckayv

Copy link
Copy Markdown
Contributor

Needs changes before merge:

  • When a reply holds more than one object with a skills array, this takes the first. For {"skills":["b"]} followed by a revised {"skills":["a","b"]}, main returns null and offers every tool, which is the safe fallback the module's header describes. This PR returns ["b"] and drops skill a's tools. Return the union of the arrays, or null, and add a test for that case.
  • Add a CHANGELOG Unreleased entry. Anthropic-configured deployments now narrow tools on replies that used to fall back to offering all of them, which is a behaviour change under the file's own rule.
  • Keep the comment pointing at server/src/routing/classify.ts. That file still uses the same greedy match.

@jiangjiang248

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review. I addressed all three points in the follow-up commit:

  • readChosenSkills now unions the known slugs from every complete object with a skills array, so a revised second object cannot drop tools named in either selection. The new regression checks both the parsed slugs and the tools actually offered.
  • Added an Unreleased changelog entry describing the behavior change for model replies containing multiple JSON objects, including Anthropic-compatible deployments.
  • Restored the comment's pointer to server/src/routing/classify.ts and noted that its greedy match is unchanged by this PR.

Validation: 40 targeted unit and integration tests passed; Biome format and lint checks passed for the changed TypeScript files; server typecheck passed. The upstream verify workflow is still awaiting maintainer approval.

davidmckayv
davidmckayv previously approved these changes Sep 26, 2026
@davidmckayv
davidmckayv enabled auto-merge (squash) September 26, 2026 19:35
@davidmckayv
davidmckayv merged commit 48b056a into CopilotKit:main Sep 26, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants