[anth-volk]

Fixes #729

Summary

• policyengine.provenance.supported_years(country_id, dataset_path) -> list[int] returns the years the installed country model supports a single-year dataset for. It builds the multi-year dataset as the model's own Simulation builds it for a single-year file, with the model's extend_single_year_dataset and its default projection end, without running any formula, and keeps the years from the dataset's data year onwards.
  • UK: data year to PROJECTION_END_YEAR, or to the end of a capital gains projection that ends earlier.
  • US: data year to the earlier of policyengine-us's default projection end (_get_default_end_year: the last year in gov.bls.cpi.cpi_u) and the last year of spm_calculator.rolling_forecast.load_forecast().years. Society-wide runs no longer accept US years before the data year. For populace_us_2024 that refuses 2022 and 2023, which the SPM forecast covers but for which the certified dataset holds almost no data (household weights sum to 57,240 in each year, against 122,537,088 in 2024, and employment income is zero). An earlier version of this PR kept those two years supported with a US_EARLY_YEAR_ALLOWANCE constant; that constant is removed.
  • The UK qualification check (uk_candidate.projected_years_from_model) and execution receipts (receipt._projected_years) now call this function, replacing the two copies of the UK rule.
• UnsupportedDatasetYearError(ValueError), exported from policyengine.provenance. Its attributes are country_id, dataset, years (the refused years) and supported_years. It keeps them through pickling. Example message:

  US dataset 'populace_us_2024' is not supported for year 2036. Supported years: 2024–2035. The installed model projects the dataset only to its supported years; policyengine-core would fill any other year by carrying the latest projected inputs forward or uprating them. Pass allow_unsupported_years=True to create year files for it anyway.

• create_datasets and ensure_datasets (UK and US) take allow_unsupported_years: bool = False, next to allow_unmanaged. create_datasets first makes every source available as a local file, then checks every requested year, and only then creates year files. ensure_datasets passes the flag through.

Results on the pinned versions

These are policyengine-uk 2.151.0, policyengine-us 2.2.1 and spm-calculator 1.0.0, in both pyproject.toml and uv.lock. The bundle manifest records the same model versions. No pins change.

Country	Dataset	Supported years
US	populace_us_2024 @ populace-us-2024-spm-20260915 (certified bytes, sha256 6496cc43…)	2024–2035
UK	microcosm_uk_2024_25, data year 2024	2024–2032

The UK figure comes from a cached microcosm_uk_2024_25.h5 of another revision, because the certified file wasn't available locally. The result depends only on the data year and the model.

Where the check runs, and its limits

• Before any year file is created. The check needs the source file, since the data year is stored only there and the bundle manifest doesn't record supported years yet. So a download comes before the check when the source is neither a local path nor already verified in data_folder.
• Existing year files are loaded without the check. ensure_datasets still decides between reusing and recreating year files from the files' stamps alone, and never opens the source on that path. Existing tests pin this behaviour. As a result, a current US year file that an earlier release created for an unsupported year (for example 2036) is still reused, and so is one created with allow_unsupported_years=True.
  • UK year files can't exist for unsupported years, because policyengine-uk only creates years in its projection.
  • Recording supported years in the bundle manifest, which is the next task, would let ensure_datasets check managed datasets before any download and on this reuse path. Raising the US year-file stamp revision would recreate the older files once, at the cost of regenerating every US year file.

Other year-file paths

• load_datasets (UK, US): loads year files that already exist. It doesn't create files or open the source, so it doesn't get the check.
• load_long_term_datasets and load_managed_long_term_datasets (US): load per-year files that the data repository projected and calibrated itself. They aren't extended from a single-year source, so this rule doesn't apply to them.
• managed_microsimulation (UK, US): constructs a simulation without a requested year. Callers can call supported_years on materialize_dataset(...).path.

Behaviour change

create_datasets and ensure_datasets used to create US year files for any year, from carried-forward or uprated inputs. They now raise for years outside 2024–2035 on populace_us_2024, including 2022 and 2023. For the UK, a requested year outside the projection now raises this error instead of No dataset found for year ....

Tests

• tests/test_dataset_years.py (new) covers:
  • the UK rule and the US rule against stand-in models, including the cap at the SPM forecast's last year, the policyengine-us default projection end, the refusal of years before the data year, and that the rule adds no year the projection does not hold;
  • each rule on the installed model with a tiny dataset;
  • the error's message, attributes and pickling;
  • the refusal before any year file or Microsimulation is created, including with several datasets;
  • the opt-out flag;
  • the two UK callers.
• One real-dataset test per country runs when POLICYENGINE_UK_DEFAULT_DATASET or POLICYENGINE_US_DEFAULT_DATASET names the bundle's default dataset file, and is skipped otherwise. This follows the existing POLICYENGINE_UK_CERTIFIED_DATASET pattern.
• Existing tests that mock create_datasets now accept or expect the new keyword (tests/test_release_manifests.py, tests/test_us_year_file_stamp.py).
• Run locally, by node ID or file only:
  • red commit: 43 selected, 5 failed, 38 errors;
  • updated US tests (removing the 2022–2023 years) against the previous code: 9 selected, 5 failed, 2 passed, 2 skipped;
  • green: tests/test_dataset_years.py 39 passed, 2 skipped;
  • the US real-dataset test on the certified populace_us_2024 (sha256 6496cc43…): 1 passed. The UK real-dataset test passed on the earlier commit;
  • ruff format --check . passes. ruff check . with the locked ruff 0.12.11 reports only the 64 UP038 findings that main also has, none in changed files, and ruff check --ignore UP038 . passes;
  • mypy --python-version 3.13 src/policyengine: 211 errors, one fewer than the 212 on main.

🤖 Generated with Claude Code