Skip to content

fix(skills/reporting-a-skill-gap): stop reports on gaps closed without a fix - #90

Merged
sourcehawk merged 3 commits into
mainfrom
fix/skill-gap-closed-unfixed-matches
Oct 6, 2026
Merged

sourcehawk merged 3 commits into
mainfrom
fix/skill-gap-closed-unfixed-matches

Conversation

@sourcehawk

@sourcehawk sourcehawk commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Description

Fixes #81

Step 3 of reporting-a-skill-gap had no branch for a search match that was closed without a fix, as a duplicate or as not planned. So an agent could file a duplicate of a report the maintainers already declined, or comment on a closed thread nobody reads. Now Step 3 reads each closed match's close reason and has a branch for each one: follow a duplicate to the issue it points at, and for a not-planned close, tell the user and file nothing unless the new case differs in a way the maintainers' reason does not cover.

Changes

  • Step 3 reads the close reason of each closed match with gh issue view --json stateReason, which prints COMPLETED, NOT_PLANNED, or DUPLICATE. The schema allows no reason, which prints an empty line. Then the agent takes the reason from the closing comment, and asks the user when the comment does not say.
  • New DUPLICATE branch: find the duplicated issue (GraphQL duplicateOf, or the closing comment) and take the branch for that issue.
  • New NOT_PLANNED branch: tell the user with the reason from the thread, and file nothing, also under a standing grant. A missing reproduction is not a difference.
  • The two existing closed-issue branches (update the plugin, or report a regression) now apply to COMPLETED closes only.
  • One Red flags row for the rationalization the baseline produced.
  • Step 1 now asks a GraphQL call to name the repository in its query, and keeps the repos/OWNER/REPO/... path rule for REST calls. Before, the path rule covered every gh api call and conflicted with the new duplicateOf query.

Challenges

The plain gh issue view shows a closed issue as CLOSED and does not show why, and --json has no field for the issue that a duplicate points at. So the duplicate branch uses a GraphQL query, which names the repository inside the query instead of in a repos/OWNER/REPO path. Step 1 says so.

Testing

RED and GREEN with isolated subagents that ran no commands, against simulated search results. With the old text, a not-planned close with a terse "Thanks. Closing." failed 2 of 2 times under a standing grant: one agent commented on the closed issue, and one filed a new issue whose only difference was a reproduction. With the issue's own scenario, all 4 runs passed but each said no branch fit and made up its own action. With the new text, the terse case passed 2 of 2, the issue's scenario passed, a duplicate of an open issue was followed to that issue, and a not-planned match with a real difference still produced a new issue that names the closed one. Both gh commands were run as written in the file: stateReason printed the three values on real issues, and the GraphQL query printed the duplicated issue's number for a known duplicate.

🤖 Generated with Claude Code

Step 3 gave a branch for a closed match only when a version fixed it. An issue closed as a duplicate or as not planned fit no branch. The plain `gh issue view` shows only CLOSED, so the agent also could not tell the close reasons apart. In a baseline with a standing grant, one agent commented on the declined issue and one filed a new issue. Neither told the user first that the maintainers had declined the gap. Both gave the missing reproduction as the reason.

Step 3 now reads the close reason with `--json stateReason`. A duplicate leads to the issue that it duplicates, through the `duplicateOf` field. For a gap closed as not planned, the agent tells the user and files nothing, also under a standing grant. It files a new issue only for a difference that the maintainers' reason does not cover, and the new issue names the closed one. A new Red flags row answers the "it had no reproduction" reason.

Fixes #81

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 09:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The duplicate lookup uses a nonexistent GraphQL field, preventing the new branch from finding the canonical issue.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Updates skill-gap reporting to handle issues closed as duplicates or not planned.

Changes:

  • Reads closed-issue reasons.
  • Adds duplicate and declined-report branches.
  • Adds a corresponding red-flag entry.
File Description
skills/​reporting-a-skill-gap/​SKILL.md Defines handling for closed matching issues.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread skills/reporting-a-skill-gap/SKILL.md

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The GraphQL command conflicts with the skill’s existing repository-scoping rule.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread skills/reporting-a-skill-gap/SKILL.md
…s repository

Step 1 asked for the repository in the path of every `gh api` call. A GraphQL call has no such path, so the rule conflicted with the `duplicateOf` query in Step 3, and an agent could reject or rewrite that query. Step 1 now asks for the path in a REST call, and for `repository(owner:"OWNER",name:"REPO")` in the query of a GraphQL call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 09:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Nullable close reasons still leave historical closed issues without a defined branch.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Handle nullable stateReason for historical closed issues

skills/​reporting-a-skill-gap/​SKILL.md:48

stateReason is nullable, including for historical closed issues, so this command can print null; that case matches none of the branches below and recreates the undefined behavior this change is intended to remove. Handle null explicitly (for example, inspect the closing context and stop for user clarification when the disposition cannot be determined) rather than asserting that every closed match yields one of these three values.

…rded close reason

GitHub's schema allows `stateReason` to be null, and then `--jq .stateReason` prints an empty line, which matched no branch of Step 3. The agent now takes the reason from the closing comment, and asks the user which branch applies when the comment does not say.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 09:31
@sourcehawk

Copy link
Copy Markdown
Owner Author

Round 3 (review 5426410916 on 92c6db2). The review posted no inline comments. Its body has one finding under "Previously missed", with no thread, so its disposition is here.

  • SKILL.md:48, "Handle nullable stateReason for historical closed issues": applied in part, in a0995ef. The schema types stateReason as a nullable enum, and a null value prints an empty line, which matched no branch. Step 3 now says: when the line is empty, take the reason from the closing comment, and ask the user which branch applies when the comment does not say. I did not take the claim about historical issues. The 400 oldest closed issues of twbs/bootstrap, rails/rails, jquery/jquery and mozilla/pdf.js all have a reason (399 COMPLETED, 1 NOT_PLANNED). An isolated subagent with the new text got an empty reason and a closing comment of "Fixed in 0.6.0". It took the regression branch from the comment, and did not ask the user.

Comment gate (addressing-review-feedback, review base 2b1c33c): 0 flagged, 0 listed, 0 removed, 0 not checked. This PR changes no code comments.

The PR body's Changes list now names the empty-reason fallback.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused guidance changes fully address issue #81 without introducing unresolved problems.

Review effort: Balanced
Findings: None

@sourcehawk
sourcehawk merged commit d0d3f88 into main Oct 6, 2026
6 checks passed
@sourcehawk
sourcehawk deleted the fix/skill-gap-closed-unfixed-matches branch October 6, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reporting-a-skill-gap: no branch for a match that was closed without a fix

2 participants