Skip to content

fix(knowledge): resume the member resurrection walk from a persisted cursor - #8365

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/member-resurrection-cursor
Sep 28, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/member-resurrection-cursor

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • The member resurrection pass walked a connector's documents from the first id on every run. A walk the deadline cut short never reached the tail, so documents past that point could stay tombstoned indefinitely
  • walkReconciliationWindows now takes startAfterId and returns { finished, lastId }. The resurrection pass resumes from a new nullable column, knowledge_connector.member_resurrection_cursor, and clears it once the tail is covered. The cursor is written once per run, only when it moved, under the member lease
  • It's a separate column rather than a reuse of member_tombstone_cursor, so an app rollback never misreads a resurrection position as a tombstone cursor
  • Purge still waits for a finished walk. Purge candidates have no observation and resurrectable documents do, so the two sets never overlap
  • withoutSecret strips the new column like the tombstone cursor
  • Migration: a single nullable ADD COLUMN with no default and no rewrite. It will be renumbered against improvement(tables): bump rows version once at commit and serialize same-value unique writes #8335 at merge
  • Responds to a review thread on v0.9.4: db contention fixes, databricks genie, snowflake cortex, additional search connectors #8356

Type of Change

  • Bug fix

Testing

  • New integration test (real Postgres): a deadline-cut first run persists the cursor at the last handled id, the resumed run covers exactly the remaining tail, and it clears the cursor. The test fails on the pre-fix code
  • Type-check, check:migrations, and the schema-mock check pass

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 28, 2026 1:50am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 10 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Critical risk] Adds database column and changes document resurrection walk logic.

The PR appears safe to merge; no actionable regression was established.

Summary

The PR persists a separate cursor for deadline-interrupted member resurrection walks, resumes after the last covered document, and clears the cursor on completion. It also updates the schema, migration, connector-row filtering, and integration coverage.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Load resurrection cursor] --> B[Scan documents after cursor]
  B --> C{Deadline reached?}
  C -- Yes --> D[Save last covered ID]
  C -- No, tail reached --> E[Clear cursor]
  E --> F[Consider purge]
Loading

Reviews (1) · Last reviewed commit: "chore(knowledge): brace the resurrection..."

@waleedlatif1
waleedlatif1 enabled auto-merge (squash) September 28, 2026 02:09
@waleedlatif1
waleedlatif1 merged commit dbeb732 into staging Sep 28, 2026
32 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/member-resurrection-cursor branch September 28, 2026 02:09

This branch was successfully deployed

1 active deployment
Preview — f6a7a3c8 Deployed Sep 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant