Repository navigation
feat!: add InboxCleaner and drop the Inbox cleanup methods - #5
Merged
Merged
Conversation
InboxCleaner runs a whole cleanup run: one cutoff from the database clock, then batches in their own READ COMMITTED transactions until one deletes nothing. Inbox.cleanup_cutoff and Inbox.delete_expired_batch are removed; Inbox now only claims.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Services currently each write their own cleanup use case around
Inbox.cleanup_cutoffandInbox.delete_expired_batch. This moves that loop into the package asInboxCleaner, so a cron entry point only builds the cleaner and callscleanup().Breaking:
Inbox.cleanup_cutoffandInbox.delete_expired_batchare removed, soInboxonly claims. The package is at 0.1 with no outside users, so this ships as 0.2.0 without a deprecation period.Design decisions
Inbox.Inboxruns on the caller's session and never commits. If it committed per batch, it could commit the caller's unrelated business writes.InboxCleanertakes anAsyncEngine, opens its own connections and never disposes the engine.cleanup()with nothing.retentionis required because only the service knows how long its producers redeliver.batch_sizedefaults to 1000.engine.execution_options()did not override an isolation level already set on the caller's engine, and a test caught that.sqlalchemy_inbox.cleaner: DEBUG per batch with a running total, so a failed run still shows how far it got, and INFO at the end. Nothing is logged on failure, since the exception already reaches the caller.Also in this PR:
docs/adr/0001-postgres-only.mdrecords why the package stays PostgreSQL-only.InboxCleaner.Tests
tests/test_cleaner.pyreplacestests/test_cleanup.py. Tests that need a batch to stop after its delete use an AFTER DELETE trigger that waits on an advisory lock the test holds. They cover:claimthat waits for an uncommitted batch, then inserts its marker.just test --cov=.: 42 passed, 100% coverage. The new file passed 8 repeated runs.Follow-ups: