Repository navigation
Make CI checks truthful and harden benchmark publication - #111
Merged
Merged
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #110
konard
changed the title
konard
marked this pull request as ready for review
October 5, 2026 12:01
Member
Author
Working session summaryCompleted in PR #111, marked ready.
Two cancelled historical jobs had no downloadable logs; their API errors are preserved. This summary was automatically extracted from the AI working session output. |
Member
Author
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost estimation:
📊 Context and tokens usage:
Total: (890.7K + 17.8M cached) input tokens, 199.7K output tokens, $10.121516 cost 🤖 Models used:
📎 Log file uploaded as Gist (14054KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
Member
Author
🎉 Auto-mergedThis pull request has been automatically merged by hive-mind.
Auto-merged by hive-mind with --auto-merge flag |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI could publish partial or stale benchmark results, suppress cancelled/failed outcomes, and run an unsupported C++ build. This PR validates complete current-run measurements, makes failures explicit, and applies the applicable practices from all three templates while preserving the Rust/C# harnesses and both C# SQLite providers.
Changes
dev/log/issues/110/pulls/111. Package raw evidence with a per-file SHA-256 integrity check to keep the PR within review-service diff limits.Reproduction and validation
The original reporter accepted empty/invalid measurements and valid baselines with missing Doublets variants. Regression logs demonstrate these failures before the fix. Git integration tests reproduce concurrent publication and changed-source rejection; artifact tests reject missing/stale/extra files and inconsistent merge trees. YAML policy tests preserve privileged-event and cancellation behavior.
Local validation passes: 61 Python tests (including six archive-integrity regressions), 9 Rust tests, 37 C# tests, provider-report checks, and eight real benchmark cells at size 1,000. Formatting, strict builds, Ruff/mypy/Bandit, actionlint/ShellCheck/zizmor, Markdown/links, secretlint, and Python/all-Rust/all-NuGet dependency audits pass. All hosted implementation workflows pass, including Rust/C# on Ubuntu, macOS and Windows. The oversized raw-evidence diff later exposed a Codacy comparison timeout; the evidence is now packaged and verified without changing its bytes. All five workflows, Codacy and CodeFactor pass on the final implementation revision
6eff47b. Archived run/check metadata identifies exact revisions; the final evidence-only revision is checked again at its own head.Evidence and upstream reports
Timeline, causes, requirements and best-practice decisions, solution choices and plans, and complete evidence archive.
Reported C# template #65, doublets #68, and Bandit #1477; added a current-version reproduction to Bandit #1041 and linked the existing Rust template #180. Historical toolchain failures were already fixed; service-owned CA warnings and practical benchmark budgets are documented with their evidence. The old human-author warning persists in
pull_request_targetlogs until this workflow change reaches the default branch.Closes #110.
Changes
.codacy.yml.github/actions/prepare-ci/action.yml.github/dependabot.yml.github/workflows/AutoMerge.yml.github/workflows/benchmarks.yml.github/workflows/cpp.yml.github/workflows/csharp.yml.github/workflows/dependencies.yml.github/workflows/quality.yml.github/workflows/rust.yml.github/zizmor.yml.gitignore.markdownlint.json.pre-commit-config.yaml.secretlintrc.jsonCONTRIBUTING.mdcpp/README.mddev/log/issues/110/pulls/111/ANALYSIS.mddev/log/issues/110/pulls/111/PLAN.mddev/log/issues/110/pulls/111/README.mddev/log/issues/110/pulls/111/SOLUTION_PLANS.mddev/log/issues/110/pulls/111/collection.logdev/log/issues/110/pulls/111/diagnostic-index.jsondev/log/issues/110/pulls/111/manifest.jsondev/log/issues/110/pulls/111/raw-evidence.tar.gzexperiments/ci/bandit_loader_probe.pyexperiments/ci/bandit_nosec_probe.pyexperiments/ci/collect_evidence.pyexperiments/ci/index_evidence.pyexperiments/ci/package_evidence.pyexperiments/ci/run_smoke.pyexperiments/ci/test_package_evidence.pyexperiments/split_store_delete/src/main.rsexperiments/system_data_sqlite/check_reports.pypyproject.tomlscripts/audit_dotnet.pyscripts/benchmark_ci.pyscripts/benchmark_report.pyscripts/check_pipeline_status.pyscripts/ci_checks.pyscripts/ci_process.pyscripts/publish_benchmarks.pyscripts/requirements-ci.txtscripts/test_audit_dotnet.pyscripts/test_benchmark_ci.pyscripts/test_benchmark_report.pyscripts/test_check_pipeline_status.pyscripts/test_ci_checks.pyscripts/test_ci_process.pyscripts/test_publish_benchmarks.py