Skip to content

feat!: add InboxCleaner and drop the Inbox cleanup methods - #5

Merged
lesnik512 merged 1 commit into
mainfrom
feat/inbox-cleaner
Oct 29, 2026
Merged

lesnik512 merged 1 commit into
mainfrom
feat/inbox-cleaner

Conversation

@lesnik512

@lesnik512 lesnik512 commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Services currently each write their own cleanup use case around Inbox.cleanup_cutoff and Inbox.delete_expired_batch. This moves that loop into the package as InboxCleaner, so a cron entry point only builds the cleaner and calls cleanup().

cleaner = InboxCleaner(engine=engine, table=INBOX_TABLE, retention=datetime.timedelta(days=7))
result = await cleaner.cleanup()  # CleanupResult(cutoff=..., deleted_count=...)

Breaking: Inbox.cleanup_cutoff and Inbox.delete_expired_batch are removed, so Inbox only claims. The package is at 0.1 with no outside users, so this ships as 0.2.0 without a deprecation period.

Design decisions

  • A separate class, not a method on Inbox. Inbox runs on the caller's session and never commits. If it committed per batch, it could commit the caller's unrelated business writes. InboxCleaner takes an AsyncEngine, opens its own connections and never disposes the engine.
  • All configuration goes in the constructor, and arguments are keyword-only. A DI provider holds the settings and the cron entry point calls cleanup() with nothing. retention is required because only the service knows how long its producers redeliver. batch_size defaults to 1000.
  • One cutoff per run, taken from the database clock in its own transaction. Every batch reuses it, so a run under constant inserts still ends.
  • One transaction per batch, stopping when a batch deletes 0. Batches that committed stay deleted if a later batch fails or the task is cancelled. The batch in progress rolls back and the error propagates.
  • No retries. SKIP LOCKED means a batch never waits on or deadlocks with a consumer, and at READ COMMITTED a serialization failure can't happen, so the remaining failures are connection-level. Batches are idempotent, so the next cron run is the retry. This also keeps db-retry a dev dependency.
  • READ COMMITTED on every batch, whatever the engine default. It is set on each connection: setting it with engine.execution_options() did not override an isolation level already set on the caller's engine, and a test caught that.
  • Concurrent runs are allowed. SKIP LOCKED splits the rows between them. An advisory lock would add a lock key to name and document without making the result more correct.
  • Logging goes to sqlalchemy_inbox.cleaner: DEBUG per batch with a running total, so a failed run still shows how far it got, and INFO at the end. Nothing is logged on failure, since the exception already reaches the caller.

Also in this PR:

  • docs/adr/0001-postgres-only.md records why the package stays PostgreSQL-only.
  • CONTEXT.md gets the Cleanup run term.
  • AGENTS.md gets the transaction-ownership rule for InboxCleaner.
  • The README cleanup section shows a one-shot cron entry point.

Tests

tests/test_cleaner.py replaces tests/test_cleanup.py. Tests that need a batch to stop after its delete use an AFTER DELETE trigger that waits on an advisory lock the test holds. They cover:

  • cancellation partway through, including the connection going back to the pool;
  • a failed batch after one that committed;
  • the cutoff staying fixed across batches;
  • three concurrent runs;
  • the isolation level being pinned;
  • a claim that waits for an uncommitted batch, then inserts its marker.

just test --cov=.: 42 passed, 100% coverage. The new file passed 8 repeated runs.

Follow-ups:

  • Removing the per-service cleanup use cases happens in those repos.

InboxCleaner runs a whole cleanup run: one cutoff from the database
clock, then batches in their own READ COMMITTED transactions until one
deletes nothing. Inbox.cleanup_cutoff and Inbox.delete_expired_batch
are removed; Inbox now only claims.
@lesnik512
lesnik512 merged commit 56ed38f into main Oct 29, 2026
13 checks passed
@lesnik512
lesnik512 deleted the feat/inbox-cleaner branch October 29, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant