Skip to content

chore: gate pull requests on 80% patch coverage - #19173

Open
yihua wants to merge 4 commits into
apache:masterfrom
yihua:ci-patch-coverage-gate
Open

yihua wants to merge 4 commits into
apache:masterfrom
yihua:ci-patch-coverage-gate

Conversation

@yihua

@yihua yihua commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Describe the issue this Pull Request addresses

There is no CI gate on the test coverage of the code a PR changes. Overall project coverage is tracked in Codecov, but a PR can add or modify code with little or no test coverage and still pass CI. This adds an enforceable minimum on changed-line (patch) coverage, evaluated and reported on the PR itself before merge.

Summary and Changelog

Adds a new Patch Coverage GitHub Actions workflow (.github/workflows/patch_coverage.yml):

  • Runs directly on the pull request (pull_request: opened / synchronize / reopened), so it evaluates the PR's own changed-line coverage and reports on the PR before merge.
  • Skips immediately (passes) when the PR changes no main-source .java/.scala lines, so docs/config/test-only PRs are not blocked and do not wait on CI.
  • Otherwise waits for the Java CI matrix to finish uploading and for Codecov to finish aggregating this PR's coverage (polls the Codecov commit endpoint until state=complete and the session count stops growing), then reads the changed-line (diff) coverage. This avoids scoring a partially merged report from the multi-job matrix and failing PRs spuriously. If Codecov does not settle within the time budget it fails open.
  • Fails the check when changed-line coverage is below 80%; passes at or above it. Threshold is a single MIN_PATCH_COVERAGE env var.
  • Gates only the lines the PR adds or modifies (patch coverage), never overall project coverage, so a PR is not penalized for the repository's existing coverage.
  • Reports the result to the run summary and, best-effort, as a PR comment.

Notes:

  • The gate is the job's own pass/fail, which shows as a check on the PR and works for fork PRs too. Posting the PR comment requires a write-scoped token, which GitHub does not grant to pull_request runs from forks; in that case the result is still shown via the check and the run summary (the comment is skipped).
  • A lighter-weight native alternative is Codecov's own coverage.status.patch target in .codecov.yml; this workflow is used instead so the gate is a first-class GitHub Actions check. .codecov.yml is intentionally left untouched here to avoid conflicting with the in-flight Codecov config modernization.
  • To make this a merge blocker, add the Patch Coverage check to branch protection required checks (separate repo-admin step).

Impact

CI-only. Adds one job per pull request that changes main source. No change to build or test behavior. Becomes a merge blocker only if the check is marked required in branch protection.

Risk Level

low. New workflow only; no existing workflow or build step is modified. It fails open if Codecov is unavailable, and does not alter test execution.

Documentation Update

none

Validation

Validated end-to-end against two throwaway PRs (both closed), each carrying this workflow plus a small hudi-utilities probe:

An earlier iteration scored while CI was still running (Codecov reported state=complete after a single upload), reading a partial 0% and failing #19174 spuriously; the workflow now waits for the Java CI run to complete and for the Codecov session count to stabilize before scoring.

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@github-actions github-actions Bot added the size:M PR with lines of changes in (100, 300] label Jul 3, 2026
@yihua
yihua force-pushed the ci-patch-coverage-gate branch 2 times, most recently from 78f5fd1 to 8eea756 Compare July 7, 2026 01:29
@yihua yihua changed the title ci: gate pull requests on 80% patch coverage chore: gate pull requests on 80% patch coverage Jul 7, 2026
@yihua
yihua marked this pull request as ready for review July 30, 2026 19:44

@hudi-agent hudi-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

No reviewable code files in this PR.

cc @yihua

yihua added 4 commits July 30, 2026 23:50
Add a Patch Coverage workflow that runs after Java CI uploads coverage to
Codecov, reads the PR's changed-line (diff) coverage from the Codecov API,
and posts a failing commit status when it is below 80%. This enforces test
coverage on new and modified code without penalizing a PR for the
repository's existing overall coverage. Docs/config/test-only PRs with no
measurable changed source lines pass automatically.
- Poll the Codecov commit endpoint until state=complete and the session
  count stops growing before reading diff coverage, so the gate does not
  score a partially-merged report from the multi-job CI matrix and fail
  PRs spuriously. Fail open if Codecov never settles within the budget.
- Post (and update in place) a patch-coverage report comment on the open
  PR so the result is visible before merging, in addition to the
  coverage/patch commit status.
Switch the trigger from workflow_run to pull_request so the guard runs on
the PR itself (including the PR that introduces it), instead of loading the
definition from the default branch. The job's own pass/fail is the gate and
the result is written to the run summary, so it works for fork PRs where the
token is read-only; the PR comment is posted best-effort on top. Skips
immediately when no main-source (.java/.scala) lines changed, and waits for
Codecov to finish aggregating the CI matrix uploads before scoring.
The gate runs concurrently with Java CI and Codecov reports state=complete
after every individual upload, so scoring on state+session-stability alone
could read a 1-session partial report and fail spuriously (observed: 0%
patch coverage while the coverage-producing test jobs were still running).
First block until the Java CI run for this head SHA is completed (all
uploads submitted), then require Codecov's session count to hold steady
across consecutive polls before reading diff coverage. Bump the budget to
cover the full matrix plus Codecov merge.
@yihua
yihua force-pushed the ci-patch-coverage-gate branch from 8eea756 to cc123a9 Compare September 10, 2026 18:08

@hudi-agent hudi-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

No reviewable code files in this PR.

cc @yihua

@hudi-bot

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M PR with lines of changes in (100, 300]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants