Skip to content

feat: explicit pydantic schema for the pipeline config - #42

Merged
mpvginde merged 5 commits into
mlwp-tools:refactor/alignmentfrom
leifdenby:feat/config-schema
Oct 27, 2026
Merged

mpvginde merged 5 commits into
mlwp-tools:refactor/alignmentfrom
leifdenby:feat/config-schema

Conversation

@leifdenby

@leifdenby leifdenby commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Addresses bullet 2 of my notes on #30: "the config.py in utils implies a config schema, should we make one explicit?"

Note

This builds on #41. Both PRs come from my fork, and GitHub can't stack PRs across forks, so this one targets refactor/alignment and also shows #41's commits. Only the last two commits are new here. Once the others are merged, the diff will show only this PR's changes.

Changes

  • mxalign/pipeline/config.py: the untyped Config dict wrapper is replaced by pydantic models, PipelineConfig plus one model per section (DatasetConfig, TransformationConfig, AlignmentConfig, TimeAlignmentConfig, SpaceAlignmentConfig, AlignmentSaveConfig, VerificationConfig, MetricConfig, SaveConfig, DatesConfig). Load with load_config(path_or_dict).
    • Strict sections reject unknown keys. Examples: a typo'd top-level section, alignment.time.return_as, or alignment.save.datasets: some.
    • Sections whose extra keys are forwarded to a function (loader, transformation, metric, save) accept them, and they're available as .kwargs.
    • dates → files expansion now happens in the validator, as before. A per-dataset dates overrides individual fields of the top-level one, and incomplete dates give a clear error. files may also be a list of patterns.
  • Runner reads typed attributes instead of .pop()/.get() on dicts. This also fixes align() mutating the config: it popped reference, so calling it twice failed. verify() is skipped when there's no verification section.
  • Adds the pydantic>=2 dependency. The uv.lock change only adds pydantic and its three dependencies.
  • New tests/test_config.py: examples/config.yaml validates, plus tests for unknown and missing keys, pass-through kwargs, and dates override and validation. The existing runner/dates tests are updated. The "unknown alignment.time option" check now comes from the schema.
  • README, CHANGELOG and examples/advanced.ipynb updated: runner.config is now the model, not a callable.

Behaviour changes worth checking

  • A config without alignment is now rejected at load time. The runner already required it.
  • inputs values of a metric must be forecast or reference. Previously any value other than reference was silently treated as forecast.
  • alignment.space: {} now runs space alignment with default options. Previously an empty dict skipped it.

100 tests pass.

🤖 Generated with Claude Code

leifdenby and others added 4 commits October 27, 2026 15:33
The YAML-driven pipeline (cli, runner, verification, transformations
and the config/dates/save helpers it alone uses) now lives in
mxalign.pipeline. The console script points at mxalign.pipeline.cli:main.
Public re-exports from mxalign/__init__.py are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the untyped Config dict wrapper with pydantic models
(PipelineConfig and one model per section) loaded via load_config().
Unknown or missing keys now fail when the Runner is created instead of
deep inside a run, and the Runner reads typed attributes rather than
popping keys (which also stops align() mutating the config). Keys that
are forwarded to loaders, transformations, metrics and save methods
are kept as pass-through kwargs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mpvginde
mpvginde merged commit d680c54 into mlwp-tools:refactor/alignment Oct 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants