Skip to content

refactor: outsource loading and validation - #27

Merged
mpvginde merged 13 commits into
mainfrom
refactor/decouple_loading_validation
Oct 26, 2026
Merged

mpvginde merged 13 commits into
mainfrom
refactor/decouple_loading_validation

Conversation

@mpvginde

@mpvginde mpvginde commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

This PR removes loading and validation of datasets from the mxalign package.

The validation of datasets is now outsourced to mlwp-data-specs
The loading of datasets is now outsourced to mlwp-data-loaders

mpvginde and others added 7 commits May 12, 2026 08:57
* port ifs-forecast loader

* updated ifs forecast loader to also handle ensemble data (#19)

---------

Co-authored-by: nielscarlier <136123206+nielscarlier@users.noreply.github.com>
Co-authored-by: Stagiair user Niels Carlier <nicarlier@kili.oma.be>
@mpvginde
mpvginde requested a review from leifdenby June 8, 2026 10:33
@leifdenby leifdenby added this to the v0.2.0 milestone Jun 19, 2026
@mpvginde mpvginde mentioned this pull request Jun 22, 2026
@mpvginde

Copy link
Copy Markdown
Contributor Author

This will temporarily break the example configs. But these will be fixed in the follow-up PR #30.

@mpvginde
mpvginde merged commit 07fdc94 into main Oct 26, 2026
2 checks passed
@mpvginde mpvginde modified the milestones: v0.3.0, v0.2.0 Oct 28, 2026
mpvginde added a commit that referenced this pull request Oct 28, 2026
#47)

* Remove leftover loaders package and finish loader separation (for v0.2.0)

PR #27 moved dataset loading and validation to mlwp-data-loaders but left a
broken src/mxalign/loaders/ package behind. Remove it and finish the
separation for the v0.2.0 release.

- Delete src/mxalign/loaders/ and drop it from __all__
- Runner.load_datasets now calls mlwp_data_loaders.load_and_validate_dataset;
  grid_mapping is applied afterwards via ds.mx.add_grid_mapping
- loader: in YAML configs is now a dotted module path or a .py file
- Update README, example config and changelog (with migration note)

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants