Skip to content

Move the simulation executor to policyengine.py 6.14.0 and keep accepting US 2022 and 2023 - #1013

Draft
anth-volk wants to merge 1 commit into
mainfrom
keep-us-2022-2023-on-py-6.14
Draft

anth-volk wants to merge 1 commit into
mainfrom
keep-us-2022-2023-on-py-6.14

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #1012

Summary

Moves the simulation executor from policyengine.py 6.12.0 to 6.14.0, and keeps the simulation service accepting US 2022 and 2023.

Since 6.13.0 (PolicyEngine/policyengine.py#730), ensure_datasets raises UnsupportedDatasetYearError when it has to create a year file for a year the installed country model doesn't support the source dataset for. For the US, that includes 2022 and 2023, which come before the 2024 data year. The Public API still offers those years for society-wide runs, so without an opt-out those runs would fail partway through. 6.12.0's ensure_datasets has no allow_unsupported_years argument, so the opt-out and the version change ship together.

The opt-out is temporary. Once the Public API stops offering US 2022 and 2023 for society-wide runs and refuses them with a 400, this repository removes it.

This replaces #1011 (6.14.0), #1002 (6.13.0) and #990 (6.12.1) for the version change. They are left open.

Changes

  • projects/policyengine-simulation-executor/pyproject.toml: policyengine[models]==6.14.0 in dependencies and in the modal-simulation-image group.
  • projects/policyengine-simulation-executor/uv.lock: regenerated with uv lock (uv 0.13.6, the release in .uv-version). Only the policyengine entry changes. Both files are identical to chore(deps): update policyengine to 6.14.0 #1011's. The bundled versions stay the same: policyengine-core 3.33.3, policyengine-us 2.2.1, policyengine-uk 2.151.0, spm-calculator 1.0.0, and the same certified data releases (US populace-us-2024-spm-20260915, UK microcosm-uk-2024-25-national-20261025T091414Z-a6299504).
  • simulation_runtime.py: _load_dataset passes allow_unsupported_years=True to ensure_datasets, with a comment saying it is temporary and when it is removed.
  • tests/test_simulation_output_dataset_selection.py: a new test, and four existing tests that compare the loader's exact arguments now expect the new one.
  • changelog.d/1012.changed.md.

Every ensure_datasets and create_datasets call

Call Years it loads Flag
simulation_runtime._load_dataset the request's year Yes (this PR)
precompute_datasets.build_dataset_impl PRECOMPUTE_YEARS: 2025, 2026, 2027 No, stays strict
precompute_datasets per-state build (source year file) the same No, stays strict
stage12_cache/precompute_datasets.build_dataset_impl PRECOMPUTE_YEARS: 2025, 2026, 2027 No, stays strict
stage12_cache/precompute_datasets per-state build (source year file) the same No, stays strict
tests/native_spm_support.py a test helper that loads real data No

_load_dataset is called from the main runtime (twice in simulation_runtime.py), the Stage 12 society-wide worker (stage12_runtime/simulation.py) and the baseline precompute (precompute_baselines._prepare_cohort_baseline). The baseline precompute therefore passes the flag too, but that changes nothing there: it loads only PRECOMPUTE_YEARS, from the year file the strict dataset precompute step built and uploaded first, and ensure_datasets loads an existing, current year file without the check.

The repository makes no direct call to create_datasets. Two tests replace it with a function that fails if called, to show that ensure_datasets reuses a prepared state file.

Other changes from policyengine.py 6.12.0 to 6.14.0

Environment variables and secrets

None added or read. The change passes a constant argument. The policyengine.py source changes from 6.12.0 to 6.14.0 add no environment variable reads and no dependencies.

Checks run locally

From projects/policyengine-simulation-executor, with uv run --frozen --extra test pytest, each selected by node ID:

  • tests/test_simulation_output_dataset_selection.py::test_load_dataset_lets_a_request_load_a_us_year_before_the_data_year (new; [2022] and [2023]): passed. Both fail with UnsupportedDatasetYearError when the opt-out is removed.
  • tests/test_simulation_output_dataset_selection.py::test_load_dataset_delegates_default_selection_to_country_loader_with_receipt ([us], [uk]): passed.
  • tests/test_simulation_output_dataset_selection.py::test_load_dataset_passes_region_dataset_name_to_managed_loader: passed.
  • tests/test_simulation_output_dataset_selection.py::test_load_dataset_maps_region_bundle_uri_to_managed_name ([True], [False]): passed.
  • tests/test_simulation_output_dataset_selection.py::test_load_dataset_reads_a_nondefault_dataset_from_the_folder_it_is_given: passed.

Also passed: Ruff 0.15.22 format --check and check on the changed Python files, python3 scripts/run_quality_guards.py, Pyright for the executor (python3 -m scripts.rules pyright projects/policyengine-simulation-executor), uv lock --check, and .github/scripts/check-changelog-fragment.sh. The rest of the test suite runs in CI.

🤖 Generated with Claude Code

policyengine.py 6.13.0 made create_datasets and ensure_datasets raise
UnsupportedDatasetYearError for a year the source dataset doesn't
support, which for the US means 2022 and 2023. The Public API still
offers those years for society-wide runs, so _load_dataset now passes
allow_unsupported_years=True until the Public API refuses them. The
precompute calls to ensure_datasets stay strict.

Fixes #1012

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move the simulation executor to policyengine.py 6.14.0 and keep accepting US 2022 and 2023

1 participant