Skip to content

Certify the Microcosm UK national from microcosm#1428 on policyengine-uk 2.164.2 (0a4c8e2c; local area 6528c865 or c9df4b86) - #731

Draft
juaristi22 wants to merge 2 commits into
mainfrom
uk-certify-microcosm-national-1428
Draft

juaristi22 wants to merge 2 commits into
mainfrom
uk-certify-microcosm-national-1428

Conversation

@juaristi22

@juaristi22 juaristi22 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

@MaxGhenis, for the UK hub: this certifies the national from PolicyEngine/microcosm#1428 on policyengine-uk 2.164.2 and supersedes #728. It needs the same decision from you as #728: which local-area cut it keeps, (a) or (b) below. That call carries over from #728, and this PR stays a draft until you make it.

The national

microcosm-uk-2024-25-national-20261031T181401Z-0a4c8e2c (microcosm_uk_2024_25):

The local area: your call

Constituency and local-authority regions route to the local-area cut (regional_release_kind: local_area). Neither cut below carries the fixes in PolicyEngine/microcosm#1419, PolicyEngine/microcosm#1420 or PolicyEngine/microcosm#1428, so area runs don't see them. The next rebuilt pair will.

(a) #725's microcosm-uk-2024-25-dense-20261027T154955Z-6528c865. This draft's default.

(b) main's microcosm-uk-2024-25-dense-20261026T105309Z-c9df4b86, the local-area data the dashboard reads today.

  • Built on policyengine-uk 2.151.0.
  • It needs its own input-surface audit against 2.164.2 before certify-data accepts it, with María as auditor (--audited-by juaristi22) and a limitation saying the cut was built on 2.151.0. Its H5 is on hand and matches the manifest pin, and the audit takes about a minute.
  • It doesn't store 25 inputs that 6528c865 stores and 2.164.2 reads: 24 alcohol, tobacco and gambling quantities, and is_apprentice. Area runs on it would read them as zero, along with the five inputs (a) defaults, so they would carry no alcohol, tobacco or gambling duty.

If you choose (b), I'll run the audit, re-certify and push it here.

Before the dashboard moves to this bundle

  • The Pension Tax Allowance row ("State Pension income tax protection") still can't run: see note 1 under the table.
  • Distributional results. policyengine-uk 2.164.2 predates Count other investment income in market and HBAI income policyengine-uk#2736, which counts other_investment_income in market and HBAI income. On this pin, household net income carries the tax on that income without the income itself. Costings read tax and benefits, so the table below is unaffected.

Budget measures on this national

Scored with #728's method: the PolicyEngine/uk-autumn-budget-dashboard-2026 measures (57095fd) and their October 2026 forecast. Each figure is the weighted change in gov_balance, in £bn. They were scored on policyengine-uk 2.162.14, as #728's table was; the bundle pins 2.164.2, and a rescore there can follow.

Columns:

Measure Year This cut #728 (4374e0bf) Published Official
Household Energy Payment 2026-27 −1.969 −1.969 −1.969 −2.625
Zero VAT on domestic electricity 2026-27 −0.788 −0.841 −0.840 −0.755
2027-28 −0.332 −0.353 −0.353 −0.305
Fuel duty rises deferred 2026-27 −0.290 −0.290 −0.301 −0.575
2027-28 −0.501 −0.501 −0.521 −0.780
State Pension income tax protection 2027-28 to 2029-30 not scored (1) not scored (1) −0.030 / −0.083 / −0.144 −0.045 / −0.090 / −0.150
Trading allowance raised to £3,000 2028-29 −0.148 −0.112 −0.118 −0.225
2029-30 −0.149 −0.116 −0.119 −0.165
2030-31 −0.148 −0.116 −0.120 −0.175
2031-32 −0.147 −0.119 −0.130 −0.180
Property income rates +2pp 2029-30 +0.402 +0.416 +0.429 +0.435
2030-31 +0.417 +0.432 +0.446 +0.445
2031-32 +0.429 +0.445 +0.459 +0.455
Savings income rates +2pp 2029-30 +0.582 +0.603 held back +0.520
2030-31 +0.611 +0.636 held back +0.555
2031-32 +0.636 +0.666 held back +0.590
Carer's Allowance earnings taper 2030-31 −0.037 −0.017 −0.024 −0.040
2031-32 −0.073 −0.032 −0.048 −0.080

(1) The dashboard's State Pension template wraps income_tax, which on policyengine-uk 2.162.x adds the Winter Fuel Payment charge, so the template refuses. The port is the dashboard's to do; the data play no part (#728, note 1).

What moved against #728's national:

Why savings sits above the costing

This corrects #728's reading, which put the excess down to ISA shifting.

  • Different base levels, about 12%.
    • This cut's 2025 tax on savings matches HMRC's Income Tax Liabilities Table 2.6 by rate: 0.98, 1.00 and 1.00. That table is HMRC's projection: £8.40bn for calendar 2025, ×1.52 on the 2023-24 outturn of £5.54bn.
    • The costing (policy costings, p. 75) takes the 2023-24 taxable savings base and grows it with the OBR's forecast of savings income: ×1.36 to calendar 2025, about £7.5bn. That base comes from HMRC's Personal Tax Model on the 2023-24 SPI, after the Personal Allowance, the Personal Savings Allowance and the starting rate.
    • With the Personal Savings Allowance and the bands frozen, taxable savings income grows faster than savings income; interest income rose 35% in 2024-25. HMRC's projection carries that, and growing the taxable base in proportion doesn't.
    • £8.40bn over £7.5bn is 1.12: the whole 2029-30 gap.
  • The lower base was tested. SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) microcosm#1428 bound the Table 2.6 rows at £7.51bn, the outturn grown on the OBR path. The calibration then couldn't meet them together with SPI Table 3.7's savings income: under the frozen-allowance rules, that income produces HMRC's level of tax. It was reverted.
  • Growth narrows the gap later. From 2028 the engine grows savings interest by its household interest income index: +3.5%, +3.0% and +3.2% a year. The costing grows by the OBR's savings forecast: +5.1%, +4.9%, +5.6% and +5.9%. So the gap falls from 12% in 2029-30 to 8% in 2031-32.
  • Behaviour and timing look small.
    • Scaled to the costing's 2025 level, this cut's 2029-30 figure is +0.521 against +0.520, which leaves little room for the costing's ISA shifting. The costing publishes no static figure, so this can't be split exactly.
    • Payment timing matters only before 2029-30: the costing's +£125m and +£555m in 2027-28 and 2028-29 carry the Self Assessment catch-up.
  • So it's a basis difference, not a data fault.

Impact

  • The UK model pin moves from 2.151.0 to 2.164.2. The data moves from the 2.151.0 pair (a6299504 and c9df4b86) to 0a4c8e2c, with (a) or (b) for the local area. UK results change with both.

Commits

  • 59662673: the national and the local area, certified by one certify-data run. It also has the lock, the release lock, the TROs, the certified test constants and the changelog fragment.
  • 6b2d0b9f: the household calculator snapshots, regenerated on 2.164.2 (PE_UPDATE_SNAPSHOTS=1). The UK households and the model surface move with the model; the US snapshots don't change.

How it was made

uv sync --frozen --extra uk --extra dev --python 3.13
uv pip install policyengine-uk==2.164.2
python scripts/audit_regional_input_surface.py \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261027T154955Z-6528c865/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --artifact microcosm_uk_2024_25_dense=<the cut's H5> --model-version 2.164.2 \
  --audited-by juaristi22 --limitation "<the limitations in the regional record>" \
  --output regional_input_surface.json
python scripts/bundle.py certify-data --country uk --data-producer populace \
  --manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261031T181401Z-0a4c8e2c/releases/microcosm-uk-2024-25-national/release_manifest.json" \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261027T154955Z-6528c865/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --model-version 2.164.2 \
  --accept-regional-model-mismatch "<the reason in the regional record>" \
  --regional-input-surface-evidence regional_input_surface.json
uvx --from uv==0.13.4 uv lock --no-config --no-sources --default-index https://pypi.org/simple --python 3.13
uv sync --frozen --extra uk --extra dev --python 3.13
python scripts/release_lock.py
python scripts/bundle.py generate --include-tros
PE_UPDATE_SNAPSHOTS=1 python -m pytest tests/test_household_calculator_snapshot.py -q
python scripts/bundle.py check
python scripts/bundle.py check --include-tros --strict-tros
python scripts/check_uk_uc_floor.py --dataset microcosm_uk_2024_25 --year 2025 --output uc_floor.json

uk_qualification.rewrite_certified_constants rewrote the certified constants in tests/test_models.py, tests/test_release_manifests.py and tests/test_uk_regions.py.

Evidence run on this head

  • The UC floor passed (check_uk_uc_floor.py, Great Britain, 2025, within 10%). The receipt holds weighted national totals only:

    • benefit units at 0.921 of DWP's figure;
    • spend at 0.951 of the OBR's.

    It took 119 s and peaked at 8.2 GB.

  • Both bundle.py check runs pass, the second with --include-tros --strict-tros.

  • Locally, 435 tests pass on the release manifests, UK regions, models, household snapshots, UC floor, bundle, TRO, certification, regional-evidence, pin and release-lock paths. The full suite runs in CI.

Disclosure

Aggregates only: weighted national totals over many records, and published HMRC, ONS, DWP, OBR and Treasury figures.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits November 1, 2026 11:18
…e2c national, 6528c865 local area)

One certify-data run: the national microcosm-uk-2024-25-national-20261031T181401Z-0a4c8e2c
certifies on its own build (policyengine-uk 2.164.2); the local-area cut
microcosm-uk-2024-25-dense-20261027T154955Z-6528c865 routes on the accepted
model mismatch, with its input surface re-audited for 2.164.2 (no dropped
column; five inputs added after 2.162.12 take their defaults). Also the
relock, the release lock, the TROs, the certified test constants and the
changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PE_UPDATE_SNAPSHOTS=1. The UK households and the model surface move with
the model; the US snapshots are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Contributor Author

Property income rates +2pp: what moved, and where the gap to the costing comes from

The table splits the measure's 2029-30 yield into its two parts:

  • The rate part: 2 points more tax on property profit outside Scotland.
  • The relief part: 2 points more finance-cost relief. Relief follows the property basic rate UK-wide, so it covers Scottish landlords too.

Measures and setup are as in the body's table: the dashboard's measure, policyengine-uk 2.162.14, change in income tax in £bn.

National Yield Rate part Relief part Landlords' finance costs, 2024-25
Published (a6299504) +0.429 +0.659 −0.231 £9.65bn
#728 (4374e0bf) +0.416 +0.653 −0.237 £10.07bn
This cut (0a4c8e2c) +0.402 +0.653 −0.251 £10.59bn
Official +0.435

What moved: the relief part.

Where the gap to the costing comes from: partnerships.

  • The costing's base (policy costings, p.74) "consists of all individuals and partnerships that receive property income", from 2024-25 Self Assessment.
  • This cut's base looks like individual landlords only.
    • It carries £32.8bn of property profit before finance costs, as bound to SPI Table 3.7, and 2.82m landlords.
    • PRIS 2024-25 gives individuals about £30.9bn and 2.85m landlords.
    • Partnerships add about £6.2bn of profit and £1.7bn of finance costs.
  • Rescaling each national's own rate and relief parts to PRIS's base for individuals and partnerships gives +0.434 for this cut. Every national lands between +0.434 and +0.444, against the official +0.435. On individuals alone they give +0.35 to +0.36.
  • So the model reproduces the costing once partnerships are in. The published national came close only because it understated finance costs, which offset the missing partnerships.
  • Is partnership property income really missing? HMRC's documentation doesn't say whether SPI's property income includes partners' shares. The totals bound it, though:
    • The SPI public use tape documentation defines it as net income from UK and overseas property, so it should exceed PRIS's individuals, who report UK property.
    • This cut's £32.8bn is £1.9bn above PRIS's individuals and £4.3bn below individuals and partnerships.
    • So at most £1.9bn of partnerships' £6.2bn could be in it, and less to the extent overseas property fills that excess.
    • Partners are taxed on their shares as property income: the SA800 partnership statement allocates UK property income to each partner. So the engine would tax them if the data carried them.
  • Caveat: PRIS doesn't split finance costs by entity, so the individuals' share is pro rata.

Bringing it in.

  • The only published source is PRIS by tax entity: Tables 1, 2 and 6 give partnerships' landlord count, receipts and expenses for 2020-21 to 2024-25. It is already in the pinned Chronicle feed.
  • No public microdata names partners. The SPI tape has one property variable, so carrying partnership property income in the national means assigning it to landlords by an imputation, controlled to the PRIS totals.
  • Finance costs together with it. Binding landlords' finance costs, which nothing binds today, belongs in the same change. Binding them alone would pull the yield further below the costing.
  • That is a candidate follow-up for PolicyEngine/microcosm.

🤖 Generated with Claude Code

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks María. You've added gross receipts to the national data, and we now estimate a £148m trading allowance cost in 2028-29.

Could we compare the savings and property estimates with the OBR's income tax lines before publishing them? Since 28 October, we've compared each estimate with the OBR line that covers what we model. We keep Table 6.1 totals as context, as we do in Tables 6 and 7 of the briefing. You use Table 6.1 totals in the Official column.

Measure Year This cut OBR line This cut against that line Table 6.1 total
Savings income rates 2029-30 +582 +443 exceeds by 31% +520
2030-31 +611 +475 exceeds by 29% +555
2031-32 +636 +510 exceeds by 25% +590
Property income rates 2029-30 +402 +394 exceeds by 2% +435
2030-31 +417 +403 exceeds by 3% +445
2031-32 +429 +412 exceeds by 4% +455
Trading allowance 2028-29 −148 −221 trails by 33% −225
2029-30 −149 −165 trails by 10% −165
2030-31 −148 −171 trails by 14% −175
2031-32 −147 −178 trails by 17% −180

Figures in £ million. We compare savings and property with income tax (measures 62 and 61), and the trading allowance with income tax plus NICs (measure 15).

For savings in 2029-30, the OBR reports £443m of income tax and £78m of onshore corporation tax, £521m in all, and the Treasury rounds that to £520m. You report a scaled £521m, which matches that total and exceeds the income tax line by £78m, or 18%.

In EFO paragraph 3.17, the OBR estimates that savers reduce the yield by around £0.2bn a year, mainly by using tax-advantaged assets such as ISAs. Could you drop "leaves little room for the costing's ISA shifting" and "ISA shifting then looks small"? If we add all £0.2bn to 2029-30 income tax, our £582m would trail the roughly £640m static estimate by about 10%. But the OBR gives no split of the reduction by tax or year, so we can't derive a static income tax line.

We need a verified reason for the 2029-30 and 2030-31 savings gaps before publishing, since both exceed our 25% bar. Our 2031-32 estimate exceeds the income tax line by 24.7%. Could you revise "Why savings sits above the costing" and #1428's savings lines against the income tax lines?

For property in 2029-30, the OBR reports £394m of income tax and £39m across onshore corporation tax and two Scottish lines. The Treasury rounds the £433m total to £435m. You report a rescaled £434m, which exceeds the income tax line by 10%; our published £429m exceeds it by 9%. The OBR estimates that landlords reduce the yield by £0.1bn a year through incorporation and non-compliance. We can use your PRIS bounds to investigate missing partnership income, but we can't quantify any omission from the rescale.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants