From ebd191e122d7e806401b53ef681e2fef1ada8930 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86gir=20M=C3=A1ni=20Hauksson?= <54936225+sourcehawk@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:49:25 +0100 Subject: [PATCH] fix(skills/copilot-review-loop): read the effort level from the review body The section `Review effort level is a precondition` said that nothing in the review payload records the effort level, and told the agent not to report that a review was balanced. That statement is false. A Copilot review in the current format states its level in a labelled line of its body, `**Review effort:** `. An agent that obeyed the skill could not tell the user a fact that the review states. The section now tells the agent to report the level from that line and to name the line as the source. If the line is absent, the agent says that the review does not state its level, and reports none. The rule against a guess from how thorough the review looks stays. Step 4 now names the body as the place where a review states its level. Copilot reviews on six pull requests of this repository show when the line is present. Each review in the current format carries the line, with or without an emoji before it, and not always as the last line. Reviews in the older format carry no such line, and one of them mentions the effort level in its prose summary. In a baseline on the unedited skill, six runs reported the level from the body. Two of the three runs on a balanced review said that the skill was wrong and did not obey it. With the new text, five runs passed: two on a balanced review, two on a body with no line, and one on a lite review. No run argued against the skill, and no run guessed a level for the body with no line. Fixes #95 Co-Authored-By: Claude Opus 5.5 --- skills/copilot-review-loop/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/skills/copilot-review-loop/SKILL.md b/skills/copilot-review-loop/SKILL.md index c53edb5..3891b35 100644 --- a/skills/copilot-review-loop/SKILL.md +++ b/skills/copilot-review-loop/SKILL.md @@ -54,7 +54,7 @@ Copilot code review runs at one of two effort levels. **Lite** is the default: f - **Interactive (standalone, or the PR-to-main gate):** when the change is substantive, say so before requesting and ask the user to select balanced — in the Reviewers selector for a one-off, or in repository settings if the repo auto-reviews on push. Then request. - **Autonomous multi-PR fan-out (sub-PR gate):** do not pause for it. The repository default governs; if a sub-PR is substantive enough that a lite review is a real gap, log it as a bubble-up concern in the state file's `## Bubble-up log` and continue. -**Nothing in the review payload records which level produced it.** Do not report that a review was balanced, and do not infer the level from how thorough the review looks. +**The review body is the only record of the level, and not every review carries it.** A review can state its level in a labelled line of its body, `**Review effort:** `, sometimes after an emoji and not always as the last line. Step 4 reads the body each round: when that line is present, report the level it states and name the line as your source. When it is absent, say that this review does not state its level, and report none. A missing line does not mean lite, a mention of effort elsewhere in the body is not the line, and how thorough the review looks is evidence of nothing. ## The loop @@ -108,7 +108,7 @@ One cycle, in order: ``` It polls `gh api .../pulls//reviews` for a Copilot review newer than `$SINCE` and exits 0 (printing the review) when one lands, or 124 on timeout. The harness re-invokes the session when it exits. Do **not** write a foreground `sleep`/`until` loop — foreground sleep is blocked and it freezes the session. -4. **Triage every finding the review carries — the comments it posted *and* the comments it suppressed** — plus any human comments already on the PR. Copilot posts some findings as inline comments and withholds others into a `Suppressed comments (N)` block in the review body; a review can carry both, or carry suppressed findings alone with nothing posted at all. Read the body every round, not only the comment threads, or the loop triages a fraction of the review it was handed: +4. **Triage every finding the review carries — the comments it posted *and* the comments it suppressed** — plus any human comments already on the PR. Copilot posts some findings as inline comments and withholds others into a `Suppressed comments (N)` block in the review body; a review can carry both, or carry suppressed findings alone with nothing posted at all. The body is also where a review states its effort level, when it states one (§Review effort level is a precondition). Read the body every round, not only the comment threads, or the loop triages a fraction of the review it was handed: ``` gh api repos/{owner}/{repo}/pulls//reviews --paginate \