[karhu]

The Stack

This stack covers refactoring the internal logic of how releases are resolved during read, download & build, in particular also in the context of parent releases / parent datasets. On top of that it refactors the public TimeNet APIs and CLI interfaces around these actions, to streamline the behavior across them.

This PR

Moves the build planner onto an ordered list of sources.

Details

A build gets each parent that the store lacks from one registry (fetch_from) or from a build (build_parents) (TNET-157). The design replaces both with one ordered list of sources, which #657 adds. This PR moves the build planner onto that list.

• plan_steps() replaces plan_parents(). It takes the InstalledBuilder of the requested release and parents_from, an ordered list of sources. For each release that the store lacks or holds damaged, resolve_release() picks the first source that provides its exact version. A registry gives a copy step, and InstalledConnectors() gives a build step.
• The steps are typed: CopyStep(release, registry, depends_on, with_payload_sources) and BuildStep(release, builder, depends_on, pinned_by, force). Step is their union. Each type has a read-only action ("copy" or "build") for the plan tree. run_plan() matches on the step type, with one arm for each type.
• A BuildStep keeps the InstalledBuilder that the lookup read, and the run builds with it. So a plan and its run read the card of each connector once.
• resolve_release() takes ask=, a function that asks one source. The planner passes one that keeps each answer.
• A release needs exactly the payload sources that its manifest lists, never the payload sources of a payload source. The Role docstring states this rule. The planner applies it before any step runs, with the walker of copy_release(), find_payload_sources() (feat(registry)!: copy a release between registries #649). The copy of a dataset brings the payload sources that its registry serves. A payload source that both the registry and the store lack gets its own step, and the copy waits for it. That copy has with_payload_sources=False. So a copy that cannot be completed fails before any download.
• The planner now makes the build step of the requested release too. A step waits for the nearest steps that it depends on, also through a release without a step.
• build_release() is now plan_release_build() and then run_plan(). run_plan() copies only through copy_release().
• BuildStep has force. A forced build step builds its release again, over any copy in the store. Two cases force the build step of the requested release: the caller asks for force, or the store holds a damaged copy of that release. So run_plan() takes no force.
• The CLI, timenet_connectors.build() and the client keep their keywords for now and map them onto parents_from. The hint for a missing parent names only the options that the build did not use. If a payload source is missing, copy_release() names both registries.
• Size: This PR has about 1580 changed lines: about 980 in the source and 600 in tests. No split keeps each part green without a later PR rewriting the code of an earlier one. Start the review with packages/timenet/src/timenet/coordinator/plan.py and build.py. They hold the planner core, about 770 of the changed lines. The rest is the callers, ask= in sources.py and the migrated tests.

Design decisions

• In one plan, the planner asks each source at most once for each release, and it keeps each answer. It takes the manifests from the answers, so the plan reads each manifest once. The run reads the manifests of the releases that it copies again, from the registry as it is at run time. A copy downloads every file of the release anyway, so the extra request makes no real difference.
• After the walk, one check covers every pin. It checks each copied release against its registry and each held release against the store. For a release to build, all readers must pin each of its files alike. So a pin conflict stops the plan before any step, also for a plan that only builds.
• The store is never a source for itself. The planner drops it from the list, also as another LocalRegistry over the same directory.
• If no other registry provides a parent or a payload source that the store holds damaged, the damaged copy stops the plan. Nothing builds over it. Only a damaged copy of the requested release gets a forced build.
• A registry that lists a release but does not serve its files does not count, so a later source can provide the release.

Breaking changes

• A parent that no installed connector builds at its exact version raises TimeNetMissingParentError, not TimeFValidationError.
• If InstalledConnectors() is in parents_from, a parent that a registry lists but does not serve now builds.

Stack

• docs: update the docs for the build coordinator #683
• feat(hints)!: give every error one hint for the SDK and one for the CLI #698
• feat(cache)!: record where each release in the local registry comes from #674
• feat(client)!: list and search every source #673
• feat(client)!: replace get() with get_manifest() and get_card() #672
• feat(build)!: build into the local registry, publish with --publish-to #671
• feat!: add timenet.build() and publish() #670
• feat(cli)!: take --local-registry, --remote-registry and --auto-build #669
• feat(config)!: make the default list of sources from three variables #668
• test: isolate every test from the real home and guard it #667
• feat(client)!: download releases into the local registry #662
• feat(client)!: take an ordered list of sources #661
• feat(client)!: read releases through the coordinator #660
• feat(coordinator)!: plan and run reads through the list of sources #659
• 👉 feat(coordinator)!: plan parents through an ordered list of sources #658
• feat(sources)!: resolve a release through an ordered list of sources #657
• feat(build)!: run every build through the coordinator #655
• feat(build)!: move the build CLI and build() onto the coordinator #654
• feat(build)!: build one prepared release in the built-in backend #653
• feat(coordinator)!: plan and run builds through build_release #652
• refactor(coordinator)!: name the plan step type Step #651
• feat(registry)!: copy a release between registries #649
• feat(registry)!: publish a local release with put_release #648
• refactor(build)!: let the connector choose the values backend #646

Changes later in the stack:

• docs: update the docs for the build coordinator #683 holds the doc changes of this PR.
• The mapping of --fetch-from, --build-parents and timenet_connectors.build(fetch_from=, build_parents=) onto parents_from is thin glue. "Defaults and the producer API" replaces it with timenet.build() and its CLI flags.
• The client's mapping in _obtain() and _prepare() is thin glue. The next PR replaces both functions with the read coordinator.

Merge message

feat(coordinator)!: plan parents through an ordered list of sources

plan_steps() takes each release that the local registry lacks from the
first source in parents_from that provides its exact version. A registry
gives a copy step, and the installed connectors give a build step.

- CopyStep and BuildStep are typed steps, and run_plan() matches on the
  type. The plan also holds the build of the requested release.
- A release needs exactly the payload sources that its manifest lists.
  The planner applies this rule before any step runs, so a copy that
  cannot complete fails before any download.
- One check covers every pin before any step runs.
- A damaged copy of a parent stops the plan unless another registry
  provides it. Only a damaged copy of the requested release gets a
  forced build.

BREAKING CHANGE: plan_parents() becomes plan_steps(), which takes an
ordered list of sources in parents_from.

BREAKING CHANGE: A parent that no installed connector builds at its
exact version raises TimeNetMissingParentError instead of
TimeFValidationError. A parent that a registry lists but does not serve
builds if a connector provides it.

[TimoStoff]

Boy that was a big one! I think I have some helpful comments, looks mostly good though I have to admit this was a bit at the limit of what I have time to review really properly….

[TimoStoff]

on second thoughs….lgtm

[claude]

I reviewed this PR and found no bugs. It is a large breaking redesign of the build planner, and new commits landed after the last approval, so a human should still look at it.

What was reviewed:

• plan_release_build and run_plan in coordinator/build.py. Every CopyStep now copies only through copy_release with with_payload_sources, and every BuildStep builds with the InstalledBuilder the lookup stored and its own force.
• The glue in client.py (_prepare, _obtain), api.py and cli.py that turns fetch_from / build_parents / auto_build into parents_from, including the new hints that name only the options the build did not use.
• The _Graph walk in plan.py. This covers source ordering, damaged local copies, payload-source steps, and the pin check, which runs once before any step.

Extended reasoning...

The change rewrites the build coordinator (plan.py, build.py), adds ask= to sources.py, changes the missing-parent message in registry/_copy.py, and updates the client, the connectors API and the CLI. It is about 1,570 changed lines across 11 files. It touches no auth, crypto or injection surface. I deferred because it is large and breaking, and it makes design decisions a maintainer should weigh: source ordering, the store never being its own source, damaged copies stopping the plan, and the error type change from TimeFValidationError to TimeNetMissingParentError. There were also commits after the reviewer's approval.

[claude]

No new blocking issues. 1 optional suggestion (a nit or a note on pre-existing code) was found and not posted. Nothing in this review needs a push before merging.

[claude]

⚠️ Code review skipped: your organization has reached its API usage limit, so Code Review can't run right now.

An organization admin can review the limit in your organization's usage and billing settings.

Once the limit resets or is raised, someone with write access to this repository can comment @claude review on this pull request to trigger a review.