Skip to content

feat: add transactional inbox ledger - #1

Merged
lesnik512 merged 2 commits into
mainfrom
feat/inbox-ledger
Oct 28, 2026
Merged

lesnik512 merged 2 commits into
mainfrom
feat/inbox-ledger

Conversation

@lesnik512

Copy link
Copy Markdown
Member

Adds Inbox, a processed-message ledger that writes its marker in the caller's own PostgreSQL transaction.

What's in it

  • make_inbox_table(metadata, table_name="inbox"): scope and message_id as the primary key, processed_at defaulting to the database's now(), and an index on processed_at. The service owns the Alembic migration.
  • Inbox(session, table): takes an AsyncSession or AsyncConnection and registers in modern-di like a REQUEST-scoped repository.
    • claim(scope=, message_id=) runs INSERT ... ON CONFLICT DO NOTHING RETURNING and returns whether this transaction inserted the marker.
    • cleanup_cutoff(retention=) reads the cutoff from the database clock.
    • delete_expired_batch(cutoff=, limit=) deletes one batch with FOR UPDATE SKIP LOCKED.
  • Every method raises RuntimeError when its session has no open transaction. A wrongly wired, uncached session therefore fails loudly instead of writing a marker that never commits.

Design decisions

  • The package never begins, commits or rolls back. Transaction ownership stays with the service, which is how db-retry's Transaction and postgres_retry already work.
  • It has no broker or DI dependency. SQLAlchemy is the only runtime requirement.
  • It's a sibling package rather than part of faststream-outbox, because faststream-outbox imports FastStream at package import and requires faststream>=0.7.6.
  • Cleanup is primitives rather than a worker. The service decides scheduling, batch size and retention per topic.

Tests

There are 34 tests against real PostgreSQL. Concurrency cases use separate connections and confirm the lock wait through pg_stat_activity. They cover:

  • first delivery, redelivery and different scopes
  • rollback after a claim
  • concurrent claims where the first transaction commits or rolls back
  • 20 concurrent deliveries producing one effect
  • the REPEATABLE READ serialization failure and its retry under postgres_retry
  • cancellation, a failed COMMIT, and a redelivery after a lost ack
  • cleanup boundaries, batch limits, skipped locked rows and replay after expiry

Locally I ran them on Python 3.14 against PostgreSQL 17 with 100% coverage. The rest of the CI matrix (3.11 to 3.15t, plus the PostgreSQL 14 floors job) hasn't run before this PR.

Not covered

  • The README examples aren't executed by a test.
  • The Kafka offset behaviour for failed handlers is documented in the README, not handled. It belongs in the services.

Inbox(session, table) records processed messages by (scope, message_id)
in the caller's transaction, with database-clock cleanup in bounded
batches. Tests run against PostgreSQL 17 and cover concurrent claims,
rollback, cancellation, commit failure and cleanup.
db-retry 0.9, a dev dependency, requires sqlalchemy>=2.0.20, so the floors
job could not install 2.0.0 and that floor was never tested.
@lesnik512
lesnik512 merged commit de6d79f into main Oct 28, 2026
13 checks passed
@lesnik512
lesnik512 deleted the feat/inbox-ledger branch October 28, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant