Skip to content

Benchmark System.Data.SQLite alongside Microsoft.Data.Sqlite and Doublets - #100

Merged
konard merged 6 commits into
mainfrom
issue-1-5af7d3ac
Oct 5, 2026
Merged

konard merged 6 commits into
mainfrom
issue-1-5af7d3ac

Conversation

@konard

@konard konard commented Sep 14, 2025 •

Copy link
Copy Markdown
Member

Fixes #1.

Adds System.Data.SQLite 2.0.4 to the current .NET 10 benchmark harness. SystemDataSQLite_Memory and SystemDataSQLite_File run the links and blog-post workloads with both 32-bit and 64-bit ids, alongside Microsoft.Data.Sqlite and every existing Doublets variant. Both SQLite providers share the same schemas, parameterized commands, immediate transactions and native e_sqlite3 library supplied by Microsoft.Data.Sqlite, so the comparison measures provider overhead against the same engine. JSON reports include both provider and engine versions.

The report compares each System.Data.SQLite variant with Microsoft.Data.Sqlite of the same durability. Existing published results retain their original measurements; subsequent Benchmarks runs include the new provider. Both READMEs document the variants and reproducible commands. The earlier unverified EF Core speedup claims have been removed from this description.

Merged current main without rewriting history and resolved its move to the new benchmark harness. Removed the obsolete root projects, exception-swallowing experiments and committed temporary SQLite files introduced by the earlier draft. Current C# CI checks the provider lifecycle and executable reports on Linux, macOS and Windows; the existing Benchmarks workflow automatically measures the new variants.

Reproduction and investigation

  • Historical CI runs 17709107773 and 17709103926 match the September 2025 draft commits. Their logs have expired (GitHub returns HTTP 410), so their exact build diagnostics are unavailable.
  • At the old PR head, the workflow's dotnet run -c Release fails with Specify which project file to use ... contains more than one project file. The maintained workflows now build and test the explicit solution under csharp/.
  • The standalone package probe initially fails with DllNotFoundException for e_sqlite3. The committed experiment reproduces this with -p:IncludeNativeSQLite=false and succeeds when Microsoft.Data.Sqlite supplies the native library. No runtime errors are swallowed.
  • Before implementation, SystemDataSQLiteVariantsAreRegistered fails for both missing variants. The report regression also fails because SystemDataSQLite_File selects SQLite_Memory as its baseline. Both pass after the change.

Verification

  • cd csharp && dotnet build -c Release -warnaserror: zero warnings/errors.
  • cd csharp && dotnet test --project SQLiteVSDoublets.Tests -c Release --no-build: 37 passing tests, including both provider lifecycles, Unicode/empty-string/date round trips, transaction rollback and command reuse, and matching native engine versions.
  • python3 experiments/system_data_sqlite/check_reports.py: four executable benchmark cases (links/objects × 32/64 bits), each with both providers in memory and files, 1,000 records and three repetitions; validates JSON versions, operation sets, sample counts and file sizes.
  • python3 -m unittest discover -s scripts -v: 18 passing report tests, including chart generation and the durability regression.
  • cd rust && cargo fmt --check && cargo clippy --release --all-targets -- -D warnings && cargo test --release: all checks pass, 9 tests pass.
  • npx --yes markdownlint-cli@0.49.1 README.md README.ru.md experiments/system_data_sqlite/README.md: passes.
  • python3 -m bandit experiments/system_data_sqlite/check_reports.py: zero findings. The smoke test uses an absolute dotnet path, fixed arguments and no shell; its two generic subprocess warnings have narrow, documented exceptions.

Final commit ce2cda2 is green: C# on Linux, macOS and Windows, all eight benchmark tables and report generation, CodeFactor and Codacy (zero new issues). The benchmark artifacts and workflow summary contain the measured System.Data.SQLite rows.

The small runs verify correctness and report integration; they do not establish a general performance advantage for either provider.

Changes

  • 16 file(s) modified
  • 273 line(s) added
  • 50 line(s) removed
  • Files:
    • .github/workflows/csharp.yml
    • README.md
    • README.ru.md
    • csharp/SQLiteVSDoublets.Tests/StoragesTests.cs
    • csharp/SQLiteVSDoublets/Harness.cs
    • csharp/SQLiteVSDoublets/Links.cs
    • csharp/SQLiteVSDoublets/Objects.cs
    • csharp/SQLiteVSDoublets/Program.cs
    • csharp/SQLiteVSDoublets/SQLiteStorage.cs
    • csharp/SQLiteVSDoublets/SQLiteVSDoublets.csproj
    • experiments/system_data_sqlite/Program.cs
    • experiments/system_data_sqlite/README.md
    • experiments/system_data_sqlite/check_reports.py
    • experiments/system_data_sqlite/system_data_sqlite.csproj
    • scripts/benchmark_report.py
    • scripts/test_benchmark_report.py

Adding CLAUDE.md with task information for AI processing.
This file will be removed when the task is complete.

Issue: #1
@konard konard self-assigned this Sep 14, 2025
- Added System.Data.SQLite NuGet package (v1.0.118) for comparison
- Created SystemDataSQLiteTestRun implementation using ADO.NET approach
- Added benchmarking comparison between System.Data.SQLite and EntityFramework
- Updated target framework to .NET 7.0 for compatibility
- Added comprehensive testing in experiments folder
- Performance comparison shows System.Data.SQLite is ~10x faster than EF Core

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@konard konard changed the title [WIP] Check out System.Data.SQLite NuGet package Check out System.Data.SQLite NuGet package Sep 14, 2025
@konard
konard marked this pull request as ready for review September 14, 2025 09:09
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

Check that everything is green and all requirements are fully delivered here.

@konard
konard marked this pull request as draft October 5, 2026 10:05
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

🤖 AI Work Session Started

Starting automated work session at 2026-10-05T10:05:12.532Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback.

Runtime: solve v2.33.11 · tool codex · model gpt-6.1-sol · task image konard/hive-mind-dind:2.33.11@sha256:999480a8daefeffc7d0ca8ca376df0252eba6a57fa7c45ab2e09b35b692b0288

@konard konard changed the title Check out System.Data.SQLite NuGet package Benchmark System.Data.SQLite alongside Microsoft.Data.Sqlite and Doublets Oct 5, 2026
@konard
konard marked this pull request as ready for review October 5, 2026 10:29
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

Working session summary

PR #100 is updated and ready for review.

  • Resolved conflicts and integrated System.Data.SQLite 2.0.4 into the current benchmarks.
  • All CI checks pass, including Linux, macOS, Windows, Codacy, and CodeFactor.
  • Local tests pass: 37 C#, 18 report, and 9 Rust tests.
  • Latest main is merged; the working tree is clean.

This summary was automatically extracted from the AI working session output.

@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Model: GPT-6.1 Sol
  • Provider: OpenAI
  • Public pricing estimate: $2.144749

📊 Context and tokens usage:

  • 162.9K / 200K (81%) input tokens, 30.3K / 128K (24%) output tokens

Total: (162.9K + 5.2M cached) input tokens, 30.3K output tokens, $2.144749 cost

🤖 Models used:

  • Tool: OpenAI Codex
  • Requested: gpt-6.1-sol
  • Thinking level: xhigh (~31999 tokens)
  • Model: GPT-6.1 Sol (gpt-6.1-sol)

📎 Log file uploaded as Gist (2866KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
konard merged commit 2c43a22 into main Oct 5, 2026
16 checks passed
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

  • All CI checks have passed

Auto-merged by hive-mind with --auto-merge flag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check out System.Data.SQLite NuGet package

1 participant