Repository navigation
Publish linear benchmark charts and complete software provenance - #114
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #113
Working session summaryImplemented all five requirements in PR #114: linear charts, generated conclusions, complete software provenance, corrected PostgreSQL formatting, and removal of the C++ placeholder. Local tests and hosted CI pass. Original measurements are preserved. Commit This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost estimation:
📊 Context and tokens usage:
Total: (209.3K + 3.5M cached) input tokens, 32.5K output tokens, $2.024174 cost 🤖 Models used:
📎 Log file uploaded as Gist (3022KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
This reverts commit 75f6dd3.
🎉 Auto-mergedThis pull request has been automatically merged by hive-mind.
Auto-merged by hive-mind with --auto-merge flag |
The generated READMEs showed only logarithmic charts and omitted Doublets/toolchain versions, while stale historical sections claimed results that the current measurements do not support. This change publishes both chart scales, records measured software provenance, and generates conclusions from the same comparisons as the tables.
Fixes #113.
Changes
doubletsfromCargo.lock,Platform.Data.Doubletsfrom the C# project, and the actual Rust compiler/.NET SDK in every future benchmark job. Recorded versions take precedence over current manifests; old reports without metadata explicitly identify what was not recorded.4e269b4and original benchmark run 37381004025: doublets 0.5.0 / Rust 1.99.0 and Platform.Data.Doublets 0.18.1 / .NET SDK 10.0.401. The reusable backfill experiment checks the log's source SHA before reading those historical manifests. Every original measurement and metadata field is preserved.Reproduction and regression coverage
Before the fix, the new tests demonstrate a missing linear chart, missing recorded library/compiler versions, missing generated conclusions, and PostgreSQL metadata outside the closing italic delimiter.
test_linear_charts_keep_small_bars_visible_and_log_charts_keep_actual_valuesinspects real Matplotlib axes and bars: a 1 ns measurement stays 1 ns on the log chart and displays as 10 ns on the linear chart with a 2,000 ns panel maximum. The input remains unchanged. Provenance tests cover historical version precedence, manifest parsing, snapshots, malformed metadata, and both languages' italic lines. A README regression verifies both committed documents exactly match regeneration and have no stale tail.Historical provenance can be recovered again with:
mkdir -p ci-logs gh run view 37381004025 --repo linksplatform/Comparisons.SQLiteVSDoublets \ --log > ci-logs/benchmarks-37381004025.log python experiments/backfill_benchmark_provenance.py docs/benchmarks/results \ ci-logs/benchmarks-37381004025.logLocal validation passes: 68 Python report/CI tests, 6 evidence tests, 9 Rust tests, and all 43 C# tests with PostgreSQL configured (none skipped). Ruff, formatting, mypy, Bandit, Rust formatting/Clippy, C# formatting/build with warnings as errors, Markdown/local links, actionlint/ShellCheck, both zizmor profiles, secretlint, and the preserved evidence integrity check pass.
Hosted validation passes on commit
d76ac6c: CI quality, all eight benchmark jobs and report generation, Codacy, and CodeFactor. The eight downloaded smoke-test artifacts validate successfully, contain measured library/toolchain snapshots, and match the latest PR merge commit and branch tree.Chart previews
C# objects, 32-bit ids, showing the absolute differences with the disclosed minimum bar height:
The accompanying log chart shows the same measurements and retains small values:
Changes
.github/workflows/benchmarks.yml.github/workflows/quality.yml.gitignoreCONTRIBUTING.mdREADME.mdREADME.ru.mdcpp/README.mdcpp/conanfile.txtdocs/benchmarks/links-csharp-32-linear.pngdocs/benchmarks/links-csharp-32.pngdocs/benchmarks/links-csharp-64-linear.pngdocs/benchmarks/links-csharp-64.pngdocs/benchmarks/links-rust-32-linear.pngdocs/benchmarks/links-rust-32.pngdocs/benchmarks/links-rust-64-linear.pngdocs/benchmarks/links-rust-64.pngdocs/benchmarks/objects-csharp-32-linear.pngdocs/benchmarks/objects-csharp-32.pngdocs/benchmarks/objects-csharp-64-linear.pngdocs/benchmarks/objects-csharp-64.pngdocs/benchmarks/objects-rust-32-linear.pngdocs/benchmarks/objects-rust-32.pngdocs/benchmarks/objects-rust-64-linear.pngdocs/benchmarks/objects-rust-64.pngdocs/benchmarks/results/links-csharp-32-100000.jsondocs/benchmarks/results/links-csharp-32-1000000.jsondocs/benchmarks/results/links-csharp-32-10000000.jsondocs/benchmarks/results/links-csharp-64-100000.jsondocs/benchmarks/results/links-csharp-64-1000000.jsondocs/benchmarks/results/links-csharp-64-10000000.jsondocs/benchmarks/results/links-rust-32-100000.jsondocs/benchmarks/results/links-rust-32-1000000.jsondocs/benchmarks/results/links-rust-32-10000000.jsondocs/benchmarks/results/links-rust-64-100000.jsondocs/benchmarks/results/links-rust-64-1000000.jsondocs/benchmarks/results/links-rust-64-10000000.jsondocs/benchmarks/results/objects-csharp-32-100000.jsondocs/benchmarks/results/objects-csharp-32-1000000.jsondocs/benchmarks/results/objects-csharp-64-100000.jsondocs/benchmarks/results/objects-csharp-64-1000000.jsondocs/benchmarks/results/objects-rust-32-100000.jsondocs/benchmarks/results/objects-rust-32-1000000.jsondocs/benchmarks/results/objects-rust-64-100000.jsondocs/benchmarks/results/objects-rust-64-1000000.jsonexperiments/backfill_benchmark_provenance.pyscripts/benchmark_provenance.pyscripts/benchmark_report.pyscripts/ci_checks.pyscripts/test_benchmark_provenance.pyscripts/test_benchmark_report.py