Skip to content

feat!: re-home source record URLs onto the upstream entry and name inlined Study after config section - #220

Merged
SkyeAv merged 6 commits into
mainfrom
update-cli-output
Oct 15, 2026
Merged

SkyeAv merged 6 commits into
mainfrom
update-cli-output

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Derived builds now place a section's source record URLs on the upstream repository's supporting sources entry (infores:pubmed-central for PMC sections) instead of the primary graph-infores entry, and the inlined supporting Study is named after the table config file plus its 1-based section index, so edge provenance names the exact repository record and configuration it came from.

Source record URL placement

  • Re-homed placement: sections without a provenance.override emit their source.url values as source_record_urls on the supporting entry matching upstream_resource_ids(provenance.repo) (Tcode._provenance_ops in src/tablassert/lib.py): infores:pubmed-central for PMC sections, infores:pubmed for PMID sections. The primary entry (the graph-level infores, e.g. infores:multiomics-kp) keeps upstream_resource_ids and gains no section record URLs: the primary is the transforming resource, not the downloadable record, the same rationale the upstream_source_record_urls override already carried.
  • Linkouts contract preserved: build-kg --linkouts keeps the per-edge https://linkouts.skyelanegoetz.com/edges/{edge_id} template on the primary entry; retrieval_sources now passes primary urls through in the upstream_urls branch, and the post-dedup sweep still resolves {edge_id} in every sources[].source_record_urls string.
  • Override sections byte-identical: override.upstream_source_record_urls re-homes exactly as specified (the primary emits nothing), override.sources emits its template verbatim, any other override keeps the legacy primary-entry emission, and --linkouts still never injects into override sections.

Study naming

  • Name: Study.name in has_supporting_studies is now <table-config.yaml> SECTION <n> (1-based position in document order) instead of the worksheet name or a source data filename like data.parquet. ingests.to_sections stamps section_index into each expanded section and Tcode carries it as a field; Study.id is unchanged (publication CURIE, or the config stem when unidentified).
  • Store keys: the stamp rides the content-addressed section hash, so duplicate identical sections in one config now get distinct caches, and pre-upgrade parquets naturally miss instead of serving stale provenance.

Docs

  • docs/configuration/table.md, docs/configuration/advanced-example.md, src/tablassert/models.py: the BaseSource.url description, the three url field rows, the sources derivation paragraph, and the Study naming prose are restated for the new behavior; CHANGELOG.md gains Unreleased Breaking Changes entries.

Testing

  • make check -> ruff clean, uv run pyright 0 errors, uv run pytest 1217 passed + 1 skipped, cargo test ok, cargo clippy --all-targets -- -D warnings clean.
  • uv run mkdocs build --strict -> pass.
  • New pins: test_tcode_collect_rehomes_pmc_record_urls_onto_pubmed_central (the reported PMC10010437 S3 xlsx case), test_tcode_collect_rehomes_pmid_record_urls_onto_pubmed, test_tcode_collect_names_study_after_config_file_and_section_index, test_tcode_collect_names_override_study_after_config_section; override byte-identity stays pinned by the existing test_production_patterns.py and test_models.py corpus.

Caveats

  • Breaking: consumers reading sources[primary].source_record_urls from derived builds must read the upstream supporting entry instead; override-authored output is untouched.
  • Section cache parquets from before this change are content-addressed and will not be reused (one-time rebuild per section).

…orting entry

Sections without a provenance.override now emit their source record URLs
on the supporting entry of the publication repository (infores:pubmed-central
for PMC sections) instead of the primary graph-infores entry. The primary
knowledge source is the transforming resource, not the downloadable record;
the --linkouts per-edge template stays on the primary entry and still
resolves via the post-dedup sweep. Override sections are byte-identical:
upstream_source_record_urls mappings re-home as specified, other overrides
keep the legacy primary-entry emission.
… index

The has_supporting_studies Study name now reads "<table-config.yaml>
SECTION <n>" (document order) instead of the worksheet or source data
filename, so edge provenance points back to the exact configuration that
produced it. ingests.to_sections stamps the 1-based section_index into
each expanded section; Tcode carries it and _provenance_ops composes the
name. Section-level direct validation is unaffected: the stamp is a
Tcode-only key, popped alongside the existing config stamp.
…utput

BaseSource.url description, the table/graph configuration reference, and the
advanced example now state that derived sections emit source record URLs on
the upstream repository supporting entry (infores:pubmed-central for PMC
sections), that --linkouts leaves only the per-edge template on the primary
entry, and that the inlined Study name is the table config file plus the
1-based section index.
- retrieval_sources Args doc now matches the upstream_urls branch, which
  passes primary urls through (the --linkouts template).
- Drop the dead study_name == study_id guard in _provenance_ops: a
  "<config> SECTION <n>" name can never equal the id.
- Rename the flattened-results local in the build loop (flat_index) so it
  cannot be confused with the new per-file Tcode.section_index.
- Pin the PMID re-homing branch and the override-section Study name in
  new tests.
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0aa02946-54e8-4b13-b3a3-6abf421d8184
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@SkyeAv SkyeAv changed the title update cli output feat!: re-home source record URLs onto the upstream entry and name inlined Study after config section Oct 15, 2026
@SkyeAv
SkyeAv merged commit 8d80f66 into main Oct 15, 2026
5 checks passed
SkyeAv added a commit that referenced this pull request Oct 15, 2026
Cut 21.0.0 and bump the package version in pyproject.toml, CITATION.cff,
and uv.lock.

Major: two breaking changes ship since 20.0.0. Derived-path builds emit
section source record URLs on the upstream repository supporting sources
entry (infores:pubmed-central for PMC sections) instead of the primary
graph-infores entry, and the inlined supporting Study is named after the
table config file plus its 1-based section index (#220); validate-infores
is folded into validate-kgx as the knowledge-source identity half, and
the standalone validate-infores command is gone (#218). Also adds no_map
on NodeEncoding to pass subject/object CURIEs through verbatim (#219) and
normalizes documentation prose to plain ASCII (#217).

Changelog:
- Versioned the Unreleased section as 21.0.0 - 2026-10-15.
- Added the missing #218 (breaking, with migration note), #219, and #217
  entries so no merged PR ships unmentioned.

Docs: none needed here; the release commit only versions existing prose.

Testing:
- make check -> ruff clean, pyright 0 errors, pytest 1239 passed + 1
  skipped, cargo test ok, cargo clippy --all-targets -- -D warnings clean
- uv run mkdocs build --strict -> pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant