Skip to content

Run the Python tests on seven shards - #2771

Open
MaxGhenis wants to merge 2 commits into
mainfrom
ci-seven-shards
Open

MaxGhenis wants to merge 2 commits into
mainfrom
ci-seven-shards

Conversation

@MaxGhenis

Copy link
Copy Markdown
Collaborator

Fixes #2770

Reproducer: policyengine_uk/tests/code_health/test_ci_test_shards.py

The Python shards use almost all of the 75-minute job limit, so a slow runner can cancel a job whose tests pass. This adds a seventh shard, the fix test-suite.yaml documents.

Why

  • Nine successful test runs started on 31 October (EDT) took 57 to 73 minutes a shard.
  • In run 41191216150, first attempt, the limit cancelled shard 1/6 with its tests at 99%, and Result failed on the cancelled job.
  • shard_durations.toml puts the listed slow tests at 134 minutes of worker time per shard on six shards and 115 on seven. The balanced split gives each of the seven shards 115 minutes.

What changed

  • test-suite.yaml: the matrix gains a 7, and a new paragraph above the job records the numbers above.
  • pr_code_changes.yaml, Makefile, docs/engineering/skills/testing.md and docs/book/validation/ubuntu-vs-macos.md: "six" and K/6 become "seven" and K/7.
  • A changelog fragment.

The job name and --shard read strategy.job-total, and the required checks name only Test / Result and Lint, so no ruleset changes.

Checks

  • test_ci_test_shards.py and test_ci_runner_variables.py pass: 126 tests.
  • Run as --shard K/7 for K = 1 to 7, the same files select 19, 23, 16, 15, 13, 22 and 18 tests, which sum to 126, and all pass.

axiom: n/a: CI configuration only

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 31, 2026 09:04
Nine complete runs on 31 October took 57 to 73 minutes a shard against
the 75-minute limit, and the limit cancelled shard 1/6 of run 41191216150
with its tests at 99%. A seventh shard brings the duration-balanced slow
tests to about 115 minutes of worker time per shard instead of 134.

Job names, outcome artifacts, Result and check_test_shards.py follow the
matrix size, so only the matrix, comments and docs change.

Fixes #2770

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 31, 2026 15:02

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python test shards reach the 75-minute CI limit

1 participant