Skip to content

Fail docgen benchmark when a quality score drops - #175

Open
jmjava wants to merge 1 commit into
mainfrom
verify/docgen-next-hole
Open

jmjava wants to merge 1 commit into
mainfrom
verify/docgen-next-hole

Conversation

@jmjava

@jmjava jmjava commented Sep 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • docgen benchmark exits 1 when a quality case scores below its baseline.
  • The packaged baseline is not rewritten.

Test plan

  • CI on this pull request passes
  • tests/test_scene_benchmark.py::test_cli_benchmark_score_regression_exits_1 is in the unit job

Made with Cursor

…ine.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant