Repository navigation
Add targeted mutation gate for scientific invariants - #175
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5403c50dec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if completed.returncode == 0: | ||
| print(f"survived: {mutant.name}", file=sys.stderr) | ||
| print(completed.stdout[-4000:], file=sys.stderr) | ||
| return False | ||
| print(f"killed: {mutant.name}") | ||
| return True |
There was a problem hiding this comment.
When pytest fails during collection, is interrupted, encounters an internal/usage error, or collects no tests, this condition treats the mutant as killed. For example, running make mutation-scientific without Hypothesis installed makes every subprocess fail while importing the copied tests, yet the script prints that all four mutants were killed and exits successfully. Accept only the pytest test-failure exit status as a killed mutant, and fail the gate for infrastructure or collection errors; a clean unmutated baseline run would also prevent false positives from pre-existing failures.
Useful? React with 👍 / 👎.
Adds a targeted mutation gate to prove the scientific tests detect known-wrong semantics.
scripts/check_mutation_sensitivity.pycopies the package into a throwaway tree, applies one source mutation, runs the scientific tests, and requires failure.make mutation-scientificruns the gate;make qualityincludes it.Local gates:
make quality(43 typed files, lint, 2133 pytest passed / 11 skipped, 4/4 mutants killed, compat39 14 passed).