Skip to content

Add Npgsql EF Core PostgreSQL object benchmarks - #99

Merged
konard merged 6 commits into
mainfrom
issue-33-e4040abb
Oct 5, 2026
Merged

konard merged 6 commits into
mainfrom
issue-33-e4040abb

Conversation

@konard

@konard konard commented Sep 14, 2025 •

Copy link
Copy Markdown
Member

Fixes #33.

Adds PostgreSQL_EFCore with Npgsql.EntityFrameworkCore.PostgreSQL 10.0.3 to the maintained .NET 10 object benchmark harness. PostgreSQL runs the same deterministic blog posts, warm-ups, fresh-store repetitions, transaction boundaries, scattered point reads and checksum-validated create/read/delete operations as the existing stores, with both 32-bit and 64-bit ids. All SQLite providers, Doublets variants and Rust/link benchmarks are preserved.

Changes

  • Implement IBlogPostsStorage<T> with EF Core saves, untracked reads and deletes by id. Clear EF tracking after each save so reads reach the server and tracking does not grow with the dataset. Store dates as the shared integer Unix seconds and allow duplicate titles.
  • Give each warm-up/repetition a generated schema and clean up only that schema, including failed lifecycles. Setup and cleanup remain outside measured operations; no caller database or shared table is deleted.
  • Enable the local provider through POSTGRESQL_CONNECTION_STRING. Runs without it retain the existing embedded defaults; explicit PostgreSQL selection without it fails with a setup message. Optional POSTGRESQL_VERBOSE=1 enables EF diagnostics without sensitive values.
  • Add provider, EF Core and PostgreSQL versions to JSON/provenance. Display server table/index/TOAST bytes separately from local files. PostgreSQL timings include ORM and client/server costs, so its row reports absolute measurements without an embedded SQLite durability ratio.
  • Run a pinned PostgreSQL 18.3 service in each C# object benchmark job, preserving comparisons on the same runner. Require its measurement during artifact validation, and add a separate PostgreSQL integration/configuration/report job to C# CI.
  • Merge current main without rewriting history and resolve the deleted-root-project conflicts by using the current harness. Update both READMEs with setup and reproducible commands; published historical measurements are preserved.

Reproduction and regression coverage

Before implementation, PostgreSQLUsesTheObjectsHarness fails for the absent variant. Three report tests also fail: PostgreSQL incorrectly selects SQLite_Memory as its baseline, server bytes are omitted, and missing PostgreSQL version metadata is accepted. They now pass. Artifact validation rejects a missing PostgreSQL result in a C# object benchmark cell.

Database tests cover both id widths and repeated validated lifecycles, Unicode/empty strings, duplicate titles, date zero, server reads after an external update, rollback/reuse and isolated schema cleanup. experiments/postgresql/check_reports.py verifies executable JSON, metadata, operation/sample sets, server bytes, all 12 embedded defaults without server configuration, and explicit configuration failures.

Validation

  • Release build with warnings as errors; maintained and experiment C# whitespace checks.
  • 43 C# tests passed against PostgreSQL 16.15; PostgreSQL executable report/configuration checks passed. Hosted integration uses PostgreSQL 18.3.
  • Existing SQLite provider executable checks passed all four category/id-width cells.
  • 59 Python tests, six evidence integrity tests and nine Rust tests passed; Rust formatting and Clippy passed.
  • Ruff, mypy, Bandit, Markdown, workflow policies, actionlint/ShellCheck, both zizmor profiles, secretlint and direct/transitive NuGet audit passed locally.

Historical failed CI runs 17704198070 and 17704191247 match the September 2025 draft commits, but their logs have expired (HTTP 410); available annotations contain only the exit status. Investigation and reproduction details document that limit and the architecture decisions. All hosted checks pass at final commit abd525c: C# and PostgreSQL integration, all eight benchmark cells and artifact/report validation, workflow and Python quality, and dependency audit. Codacy and CodeFactor also pass.

Bounded runs verify correctness and report integration; they do not establish a general performance advantage. No UI changes.

Changes

  • 16 file(s) modified
  • 622 line(s) added
  • 14 line(s) removed
  • Files:
    • .github/workflows/benchmarks.yml
    • .github/workflows/csharp.yml
    • README.md
    • README.ru.md
    • csharp/SQLiteVSDoublets.Tests/PostgreSQLTests.cs
    • csharp/SQLiteVSDoublets.Tests/StoragesTests.cs
    • csharp/SQLiteVSDoublets/Harness.cs
    • csharp/SQLiteVSDoublets/PostgreSQLBlogPosts.cs
    • csharp/SQLiteVSDoublets/Program.cs
    • csharp/SQLiteVSDoublets/SQLiteVSDoublets.csproj
    • dev/log/issues/33/pulls/99/ANALYSIS.md
    • experiments/postgresql/check_reports.py
    • scripts/benchmark_ci.py
    • scripts/benchmark_report.py
    • scripts/test_benchmark_ci.py
    • scripts/test_postgresql_report.py

Adding CLAUDE.md with task information for AI processing.
This file will be removed when the task is complete.

Issue: #33
@konard konard self-assigned this Sep 14, 2025
This commit adds PostgreSQL support to the comparison benchmarks alongside
SQLite and Doublets. The implementation includes:

- Added Npgsql.EntityFrameworkCore.PostgreSQL package dependency
- Created PostgreSQL namespace with PostgreSQLDbContext and PostgreSQLTestRun
- Updated benchmark configuration to include PostgreSQL benchmarks
- Updated Program.cs to include PostgreSQL in manual comparisons
- Upgraded target framework to .NET 7.0 for compatibility

The PostgreSQL implementation follows the same patterns as the SQLite
implementation, allowing for consistent performance comparisons across
all three data storage approaches.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@konard konard changed the title [WIP] Add a comparison with Npgsql.EntityFrameworkCore.PostgreSQL Add PostgreSQL comparison with Npgsql.EntityFrameworkCore.PostgreSQL Sep 14, 2025
@konard
konard marked this pull request as ready for review September 14, 2025 00:48
@konard

konard commented Oct 5, 2026 •

Copy link
Copy Markdown
Member Author

We need to make sure this fully fits to our benchmarking architecture and adds value, not reduce it.

@konard
konard marked this pull request as draft October 5, 2026 21:44
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

🤖 AI Work Session Started

Starting automated work session at 2026-10-05T21:44:17.808Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback.

Runtime: solve v2.33.11 · tool codex · model gpt-6.1-sol · task image konard/hive-mind-dind:2.33.11@sha256:999480a8daefeffc7d0ca8ca376df0252eba6a57fa7c45ab2e09b35b692b0288

@konard konard changed the title Add PostgreSQL comparison with Npgsql.EntityFrameworkCore.PostgreSQL Add Npgsql EF Core PostgreSQL object benchmarks Oct 5, 2026
@konard
konard marked this pull request as ready for review October 5, 2026 22:09
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

Working session summary

Updated PR #99 and marked it ready for review.

PostgreSQL EF Core now uses the shared benchmark harness for both id widths, with regression tests and separate server storage metrics. Merge conflicts are resolved, all checks pass, and the working tree is clean.


This summary was automatically extracted from the AI working session output.

@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Model: GPT-6.1 Sol
  • Provider: OpenAI
  • Public pricing estimate: $2.558980

📊 Context and tokens usage:

  • 214.4K input tokens across requests (cumulative, larger than the 200K context window, so not one request's context), 41.7K / 128K (33%) output tokens

Total: (214.4K + 5.4M cached) input tokens, 41.7K output tokens, $2.558980 cost

🤖 Models used:

  • Tool: OpenAI Codex
  • Requested: gpt-6.1-sol
  • Thinking level: xhigh (~31999 tokens)
  • Model: GPT-6.1 Sol (gpt-6.1-sol)

📎 Log file uploaded as Gist (3994KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
konard merged commit 4e269b4 into main Oct 5, 2026
25 checks passed
@konard

konard commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

  • All CI checks have passed

Auto-merged by hive-mind with --auto-merge flag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a comparison with Npgsql.EntityFrameworkCore.PostgreSQL

1 participant