Repository navigation
fix(skills/copilot-review-loop): report the effort level a review states - #96
Merged
Merged
Conversation
…w body The section `Review effort level is a precondition` said that nothing in the review payload records the effort level, and told the agent not to report that a review was balanced. That statement is false. A Copilot review in the current format states its level in a labelled line of its body, `**Review effort:** <level>`. An agent that obeyed the skill could not tell the user a fact that the review states. The section now tells the agent to report the level from that line and to name the line as the source. If the line is absent, the agent says that the review does not state its level, and reports none. The rule against a guess from how thorough the review looks stays. Step 4 now names the body as the place where a review states its level. Copilot reviews on six pull requests of this repository show when the line is present. Each review in the current format carries the line, with or without an emoji before it, and not always as the last line. Reviews in the older format carry no such line, and one of them mentions the effort level in its prose summary. In a baseline on the unedited skill, six runs reported the level from the body. Two of the three runs on a balanced review said that the skill was wrong and did not obey it. With the new text, five runs passed: two on a balanced review, two on a body with no line, and one on a lite review. No run argued against the skill, and no run guessed a level for the body with no line. Fixes #95 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused guidance matches the documented review formats and handles missing labels safely.
0 open findings
What changed in this PR
Fixes #95 by teaching the Copilot review loop to report only explicitly stated review effort levels.
Changes:
- Parses the labelled effort line from review bodies.
- Avoids inferring a level when the line is absent.
- Clarifies that Step 4 reads effort metadata from the body.
| File | Description |
|---|---|
skills/copilot-review-loop/SKILL.md |
Documents reliable effort-level reporting during review triage. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #95
The skill said that nothing in a Copilot review records its effort level, and told the agent not to report that a review was balanced. That statement is false: a review in the current format states its level in a labelled line of its body. This PR tells the agent to report the level from that line, and to report no level when the line is absent.
Changes
## Review effort level is a precondition: the agent reads the line**Review effort:** <level>in the review body, reports that level, and names the line as its source.Challenges
Not every review carries the line. Copilot reviews on six pull requests of this repository show two formats. Each review in the current format (#78, #90, #91, #94) carries the line, with or without an emoji before it, and not always as the last line. Reviews in the older format (#28, #32) carry no such line, and one body on #32 mentions the effort level in its prose summary. The rule for the absent line and the rule for a prose mention come from those reviews.
Testing
Each run gave a new agent the skill file and a review body, and asked for the triage report to the user.
On the unedited skill, six runs reported the level from the body: three on a lite review and three on a balanced review. So the failure in the issue did not occur in these runs. But two of the three runs on a balanced review told the user that the skill was wrong, and did not obey it.
With the new text, five runs passed. Two runs on a balanced review reported
Balancedand quoted the line, with no argument against the skill. Two runs on a body with no line, and with a summary that reads as thorough, said that the review does not state its level and guessed none. One run on a lite review reportedLitewith the line as its source.No script changed, so the script tests did not run. The JSON files in
.claude-plugin/parse.🤖 Generated with Claude Code