[karhu]

The Stack

This stack covers refactoring the internal logic of how releases are resolved during read, download & build, in particular also in the context of parent releases / parent datasets. On top of that it refactors the public TimeNet APIs and CLI interfaces around these actions, to streamline the behavior across them.

This PR

Adds the build coordinator, build_release(), next to the old build path. Later PRs in the stack move the callers onto it.

Details

TimeNet starts a build on five paths, and each path plans and runs the build in its own way (TNET-157). The coordinator is the one place that plans a build and runs its steps.

• timenet.coordinator.build.build_release(dataset, *, store, fetch_from, build_parents, force, pinned_by, open_access_only, options) builds one release into a local store, after each release that its build needs. It returns the Plan that ran.
• It finds the backend that claims the dataset. If the caller asks for another version than the card declares, it raises TimeNetVersionUnavailableError before anything runs.
• It plans with plan_parents(), adds the build of the requested release, and runs each step in order. A copy step uses copy_release(). A build step calls build_prepared() of the backend for that exact release. Each step sends StepStarted and StepFinished.
• The BuilderBackend protocol gets dataset_ids() and build_prepared(release, store, *, force, pinned_by, options). BuildOptions holds keep_cache, the progress callback and quiet. Unless quiet is set, an isolated child prints its shards and downloads to the inherited stderr. builders.installed_builder() returns an InstalledBuilder with the first backend whose dataset_ids() holds the ID, and the card of the dataset. builders.declared() also checks the version. It raises TimeNetDatasetNotFoundError for an unknown ID and TimeNetVersionUnavailableError for another version.
• BuildOptions.keep_cache and quiet are not read in this PR. ConnectorBuilder.build_prepared() in feat(build)!: build one prepared release in the built-in backend #653 reads them.
• ConnectorBuilder.dataset_ids() lists the IDs of the cards on disk and imports no connector. It wraps discovery.known_ids(), which the build command calls to check an ID.
• timenet-build build takes a dataset ID or an exact release, org/name@version. For another version than the card declares, it raises TimeNetVersionUnavailableError before anything runs.
• plan_parents() takes open_access_only. With it, a local store gets the same access checks as a registry that is not local. The checks cover the card and the lineage of a requested release that the store already holds.

Design decisions

• The coordinator lives in timenet and imports nothing from timenet_connectors, so any package can provide a backend. A ruff rule that bans the import makes sure of that.
• There is one version check, builders.declared(), which the coordinator and the build command call. TimeNetVersionUnavailableError is a TimeNetDatasetNotFoundError and a TimeFValidationError, so the error handlers of both current callers keep working.
• Only the requested release gets force. The coordinator does not build again a parent that the store holds intact.
• If the store holds the requested release intact, its copy step copies only each payload source that the store lacks or holds damaged.
• A dataset ID matches only the ID that a card declares. DatasetId is case-sensitive and allows _, so timenet/hello_world is a different ID from timenet/hello-world. For an unknown ID, timenet-build build still shows the usage error with the list of known IDs.
• discovery.known_ids() skips a card that cannot be read, or that has no dataset_id, with a ConnectorCardWarning that names the file. So a broken card fails only its own dataset. The card tests read every card, so CI finds such a card first.
• The coordinator is the planner that the front ends share. It is not part of the user API, so the API docs leave out timenet.coordinator.

Breaking changes

• A BuilderBackend must implement dataset_ids() and build_prepared().
• builders.find_builder() and builders.exact_builder() are gone. Use builders.installed_builder(), or builders.declared() to check the version too.
• timenet-build build and timenet_connectors.build() no longer accept the module spelling of an ID, such as timenet/hello_world. Use the ID from the card, timenet/hello-world.

Stack

• docs: update the docs for the build coordinator #683
• feat(hints)!: give every error one hint for the SDK and one for the CLI #698
• feat(cache)!: record where each release in the local registry comes from #674
• feat(client)!: list and search every source #673
• feat(client)!: replace get() with get_manifest() and get_card() #672
• feat(build)!: build into the local registry, publish with --publish-to #671
• feat!: add timenet.build() and publish() #670
• feat(cli)!: take --local-registry, --remote-registry and --auto-build #669
• feat(config)!: make the default list of sources from three variables #668
• test: isolate every test from the real home and guard it #667
• feat(client)!: download releases into the local registry #662
• feat(client)!: take an ordered list of sources #661
• feat(client)!: read releases through the coordinator #660
• feat(coordinator)!: plan and run reads through the list of sources #659
• feat(coordinator)!: plan parents through an ordered list of sources #658
• feat(sources)!: resolve a release through an ordered list of sources #657
• feat(build)!: run every build through the coordinator #655
• feat(build)!: move the build CLI and build() onto the coordinator #654
• feat(build)!: build one prepared release in the built-in backend #653
• 👉 feat(coordinator)!: plan and run builds through build_release #652
• refactor(coordinator)!: name the plan step type Step #651
• feat(registry)!: copy a release between registries #649
• feat(registry)!: publish a local release with put_release #648
• refactor(build)!: let the connector choose the values backend #646

Changes later in the stack:

• docs: update the docs for the build coordinator #683 holds the doc changes of this PR.
• ConnectorBuilder gets build_prepared() in feat(build)!: build one prepared release in the built-in backend #653. Until then, only the tests call build_release().
• feat(build)!: move the build CLI and build() onto the coordinator #654 moves the CLI and timenet_connectors.build() onto build_release(), and feat(build)!: run every build through the coordinator #655 moves the client. feat(build)!: run every build through the coordinator #655 removes knows(), declared_version() and the old build() from the protocol, and renames build_prepared() to build(). Until feat(build)!: move the build CLI and build() onto the coordinator #654, timenet_connectors.build() accepts the module spelling of an ID and then raises its documented LookupError.
• The fetch_from= and build_parents= arguments of build_release() are thin glue for the current callers. "Client reads on the coordinator" replaces them.

Merge message

feat(coordinator)!: plan and run builds through build_release

build_release() plans one build and runs its steps, so that the five
build paths can share one planner. It runs next to the old build path
for now.

- It copies each missing parent with copy_release() and builds each
  release with build_prepared() of the backend that claims it.
- Only the requested release gets force. An intact parent in the
  local registry does not build again.
- builders.declared() is the one version check. Another version than
  the card declares raises TimeNetVersionUnavailableError before
  anything runs.
- The coordinator imports nothing from timenet_connectors, so any
  package can provide a backend.

BREAKING CHANGE: A BuilderBackend must implement dataset_ids() and
build_prepared().

BREAKING CHANGE: builders.find_builder() and builders.exact_builder()
become builders.installed_builder() and builders.declared().

BREAKING CHANGE: timenet-build build and timenet_connectors.build() no
longer accept the module spelling of an ID, such as
timenet/hello_world.

[TimoStoff]

LGTM! Thank you. The main issue here again is sloptests and really hard to read Claude comments

[claude]

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

[claude]

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

Still open from earlier reviews (1):

• 🔴 packages/timenet/src/timenet/coordinator/build.py:105 — When the store already holds the requested release, a copy of a missing payload source also pulls that source's own pay…

If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.