You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The member resurrection pass walked a connector's documents from the first id on every run. A walk the deadline cut short never reached the tail, so documents past that point could stay tombstoned indefinitely
walkReconciliationWindows now takes startAfterId and returns { finished, lastId }. The resurrection pass resumes from a new nullable column, knowledge_connector.member_resurrection_cursor, and clears it once the tail is covered. The cursor is written once per run, only when it moved, under the member lease
It's a separate column rather than a reuse of member_tombstone_cursor, so an app rollback never misreads a resurrection position as a tombstone cursor
Purge still waits for a finished walk. Purge candidates have no observation and resurrectable documents do, so the two sets never overlap
withoutSecret strips the new column like the tombstone cursor
New integration test (real Postgres): a deadline-cut first run persists the cursor at the last handled id, the resumed run covers exactly the remaining tail, and it clears the cursor. The test fails on the pre-fix code
Type-check, check:migrations, and the schema-mock check pass
[Critical risk] Adds database column and changes document resurrection walk logic.
The PR appears safe to merge; no actionable regression was established.
Summary
The PR persists a separate cursor for deadline-interrupted member resurrection walks, resumes after the last covered document, and clears the cursor on completion. It also updates the schema, migration, connector-row filtering, and integration coverage.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Load resurrection cursor] --> B[Scan documents after cursor]
B --> C{Deadline reached?}
C -- Yes --> D[Save last covered ID]
C -- No, tail reached --> E[Clear cursor]
E --> F[Consider purge]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
walkReconciliationWindowsnow takesstartAfterIdand returns{ finished, lastId }. The resurrection pass resumes from a new nullable column,knowledge_connector.member_resurrection_cursor, and clears it once the tail is covered. The cursor is written once per run, only when it moved, under the member leasemember_tombstone_cursor, so an app rollback never misreads a resurrection position as a tombstone cursorwithoutSecretstrips the new column like the tombstone cursorADD COLUMNwith no default and no rewrite. It will be renumbered against improvement(tables): bump rows version once at commit and serialize same-value unique writes #8335 at mergeType of Change
Testing
check:migrations, and the schema-mock check passChecklist