Skip to content

ENG-1986 Refactor Obsidian sync to use the new cross-app schemas (3/6) - #1631

Open
sid597 wants to merge 2 commits into
eng-1986-send-obsidian-schemas-through-the-cross-app-convertersfrom
eng-1986-send-obsidian-nodes-through-the-cross-app-converters
Open

sid597 wants to merge 2 commits into
eng-1986-send-obsidian-schemas-through-the-cross-app-convertersfrom
eng-1986-send-obsidian-nodes-through-the-cross-app-converters

Conversation

@sid597

@sid597 sid597 commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Reviewer brief

ENG-1986 moves Obsidian sync off its own converters and onto the shared CrossApp converters, without loss of functionality. It ships as six PRs. PRs 1 to 4 and 6 are stacked on #1575 (ENG-1823). PR 5 targets main:

  1. ENG-1986 Refactor Obsidian sync to use the new cross-app schemas (1/6) #1620: tests that pin exactly what Obsidian sync sends to upsert_concepts today. No production change.
  2. ENG-1986 Refactor Obsidian sync to use the new cross-app schemas (2/6) #1630: node type, relation type and relation triple schemas go through the shared converters.
  3. This PR. Nodes and their Content go through the shared converters.
  4. ENG-1986 Refactor Obsidian sync to use the new cross-app schemas (4/6) #1632: relation instances go through the shared converters.
  5. ENG-1986 Refactor Obsidian sync to use the new cross-app schemas (5/6) #1634: color becomes a CrossApp field on node types and relation types, in packages/database. It doesn't depend on PRs 1 to 4.
  6. ENG-1986 Refactor Obsidian sync to use the new cross-app schemas (6/6) #1633: Obsidian sync sends color through the shared converters. It needs PR 5, and its branch carries PR 5's change until that merges.

Why six and not one: each step changes one kind of row, and #1620's tests show whether a stored value changed. PR 5 changes the shared contract, so it goes first in its own PR and needs its own review. Together the steps are well over the 400-line guideline.

  • Result: Obsidian sync builds each node's Concept and its direct and full Content with nodeInstanceToCrossApp and the shared crossAppNodeToDbConcept, the converter Roam sync already uses.
  • Result: No stored value changes, apart from the one timing change below. Node rows no longer send space_id or is_schema: false, and author_id becomes author_local_id. upsert_concepts sets space_id from v_space_id, defaults is_schema to false and resolves author_local_id to the same account. This comes from reading concept.sql. The unit tests mock the RPC.
  • Review focus: CrossAppNode.content.direct is required, so the converter always returns a direct Content. Sync sends direct only when the title changed, as before, so conceptConversion.ts drops it otherwise. full needs no filter: it is set only when the note's text changed.
  • Review focus: conceptConversion.ts sets these values after the converter, because the CrossApp types have no field for them: name (the file path), literal_content.label, literal_content.source_data, and each Content's creator_local_id and metadata.
  • Review focus: buildNodeContentsWithEmbeddings is now buildChangedNodeContent. It returns each changed node's title embedding and text, and nodeInstanceToCrossApp builds the Content. The embedding model comes from the shared converter's default, openai_text_embedding_3_small_1536, the value the removed EMBEDDING_MODEL held. The removed empty-input early return guarded nothing: with no nodes, no file is read and no request is sent.
  • Review focus: sync skips imported notes (syncDgNodesToSupabase.ts:504), so node rows no longer carry an importedFromRid branch that could never run.
  • Risk or follow-up: one timing change. A note that gets its nodeInstanceId during a sync is now sent on that sync. Before, the id was read from the cached frontmatter, which doesn't have it yet, so the row was dropped and sent on the next sync. Run on both builds against a local database: the base build sent the new note on the next sync, and this PR's build on the same one. It applies only to a node created with no other note open. With a note open, the plugin creates a relation to it, which gives the new note its id at creation, and both builds send it on the same sync.
  • Risk or follow-up: the console.error calls in the rewritten catch blocks are kept as they were.
Before After Stored value
space_id, is_schema: false not sent same, set by upsert_concepts
author_id: context.userId author_local_id same account
getNodeExtraData times from file.stat createdAt, modifiedAt from the times sync read at change detection same
source_local_id, schema_represented_by_local_id, core_title, the Source slot from the shared converter same, except the timing change above
name, label, source_data set after the converter same
direct and full Content rows from the shared converter, plus creator_local_id and metadata same, pinned by #1620

Verification

  • From apps/obsidian on this head: vitest run passes (259 tests), tsc --noEmit --skipLibCheck passes, and prettier is clean on the changed files.
  • eslint --max-warnings 0 on the changed files reports three warnings this PR didn't add: syncDgNodesToSupabase.ts:1152 and :1157 (from [ENG-1178] Automatic local remote sync nodes relations #706) and an unused e at upsertNodesAsContentWithEmbeddings.ts:33.
  • Not run on this head: pnpm ci:validate for the whole repository, and a sync against a database.

Loom video

Not recorded yet.

Scope check

  • Ran $scope-check against ENG-1986 and the final diff.
  • Scope beyond Done When: A note that gets its nodeInstanceId during a sync (a node created with no other note open) is sent on that sync, not the next one.
  • Required now: CrossAppNode.localId is required. The old read cast the id from cached frontmatter and hid a missing value.
  • Anyone affected or consulted: Not documented.
  • Decision: recorded on ENG-1986, 2026-11-16.

Standards check

  • Ran $dg-pr-adherence-check against the final diff and PR metadata.

Local delegated full review

  • Ran a comprehensive review of the entire final diff in a subagent with a fresh context. Use $dg-delegated-full-review when no other full-review workflow is available.

- nodeInstanceToCrossApp reads nodeInstanceId and nodeTypeId from
  ObsidianDiscourseNodeData instead of casting the cached frontmatter.
- Drop the importedFromRid spread for node instances: sync skips
  imported notes before they reach discourseNodeInstanceToLocalConcept.
- Rename buildNodeContentsWithEmbeddings to buildChangedNodeContent.
- The converter comment now covers every row, not only schemas.
@vercel

vercel Bot commented Nov 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
discourse-graph Skipped Skipped Nov 16, 2026 7:53am UTC

Request Review

@linear-code

linear-code Bot commented Nov 16, 2026

Copy link
Copy Markdown

ENG-1986

@supabase

supabase Bot commented Nov 16, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@sid597
sid597 added this pull request to stack #1624 November 16, 2026 08:00
@sid597
sid597 marked this pull request as ready for review November 16, 2026 08:00
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T08:03:48.177566Z 44be2ba Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

This branch was previously deployed

1 inactive deployment
Preview — 44be2ba7 Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant