Repository navigation
feat: explicit pydantic schema for the pipeline config - #42
Merged
mpvginde merged 5 commits intoOct 27, 2026
Merged
Conversation
leifdenby
force-pushed
the
feat/config-schema
branch
from
October 27, 2026 14:04
5aab7ac to
95be241
Compare
The YAML-driven pipeline (cli, runner, verification, transformations and the config/dates/save helpers it alone uses) now lives in mxalign.pipeline. The console script points at mxalign.pipeline.cli:main. Public re-exports from mxalign/__init__.py are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the untyped Config dict wrapper with pydantic models (PipelineConfig and one model per section) loaded via load_config(). Unknown or missing keys now fail when the Runner is created instead of deep inside a run, and the Runner reads typed attributes rather than popping keys (which also stops align() mutating the config). Keys that are forwarded to loaders, transformations, metrics and save methods are kept as pass-through kwargs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leifdenby
force-pushed
the
feat/config-schema
branch
from
October 27, 2026 14:33
95be241 to
5986c74
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses bullet 2 of my notes on #30: "the config.py in utils implies a config schema, should we make one explicit?"
Note
This builds on #41. Both PRs come from my fork, and GitHub can't stack PRs across forks, so this one targets
refactor/alignmentand also shows #41's commits. Only the last two commits are new here. Once the others are merged, the diff will show only this PR's changes.Changes
mxalign/pipeline/config.py: the untypedConfigdict wrapper is replaced by pydantic models,PipelineConfigplus one model per section (DatasetConfig,TransformationConfig,AlignmentConfig,TimeAlignmentConfig,SpaceAlignmentConfig,AlignmentSaveConfig,VerificationConfig,MetricConfig,SaveConfig,DatesConfig). Load withload_config(path_or_dict).alignment.time.return_as, oralignment.save.datasets: some..kwargs.dates→filesexpansion now happens in the validator, as before. A per-datasetdatesoverrides individual fields of the top-level one, and incomplete dates give a clear error.filesmay also be a list of patterns.Runnerreads typed attributes instead of.pop()/.get()on dicts. This also fixesalign()mutating the config: it poppedreference, so calling it twice failed.verify()is skipped when there's noverificationsection.pydantic>=2dependency. Theuv.lockchange only adds pydantic and its three dependencies.tests/test_config.py:examples/config.yamlvalidates, plus tests for unknown and missing keys, pass-through kwargs, and dates override and validation. The existing runner/dates tests are updated. The "unknownalignment.timeoption" check now comes from the schema.examples/advanced.ipynbupdated:runner.configis now the model, not a callable.Behaviour changes worth checking
alignmentis now rejected at load time. The runner already required it.inputsvalues of a metric must beforecastorreference. Previously any value other thanreferencewas silently treated as forecast.alignment.space: {}now runs space alignment with default options. Previously an empty dict skipped it.100 tests pass.
🤖 Generated with Claude Code