Skip to content

gh-158031, gh-158033: Fix UAF in iter/aiter sentinel comparison - #158145

Open
iliasabk wants to merge 1 commit into
python:mainfrom
iliasabk:fix-iter-sentinel-uaf
Open

iliasabk wants to merge 1 commit into
python:mainfrom
iliasabk:fix-iter-sentinel-uaf

Conversation

@iliasabk

@iliasabk iliasabk commented Sep 25, 2026 •

Copy link
Copy Markdown

Issues

Fixes #158031, fixes #158033 — two same-class heap-use-after-free reports in the callable-with-sentinel iterators added by gh-98469.

Root cause

Both calliter_iternext() (sync iter(callable, sentinel)) and acallawaitable_handle_error() (async aiter(callable, sentinel)) compare it->it_sentinel against the produced value with PyObject_RichCompareBool() while holding only a borrowed reference. The comparison can run arbitrary Python code via a custom __eq__, and that code can release the sentinel mid-comparison:

  • sync: Sentinel.__eq__ calls it.__setstate__(((), StopIteration)), which does Py_XSETREF(it->it_sentinel, NULL); once the __eq__ frame is torn down the last reference is gone and do_richcompare() continues with a freed v (ASan: UAF read at iterobject.c:285).
  • async: Sentinel.__eq__ drives ait.__anext__().__await__().send(None), which reaches acallawaitable_start → callable raises StopAsyncIteration → acalliter_exhaust() → Py_CLEAR(it->it_sentinel); same freed-v continuation (ASan: UAF read at iterobject.c:644).

Fix

Hold a strong reference to the sentinel across the PyObject_RichCompareBool() call in both functions — the same pattern already used elsewhere for callbacks that can mutate the iterator (see the ordering note in calliter_reduce, gh-101765). No behavior change: the in-flight comparison still uses the sentinel that was current when the comparison started, and subsequent state (calliter_exhaust, _PyGen_SetStopIterationValue) reads the post-mutation fields as before.

Tests

  • test_calliter_sentinel_reentrant_setstate in test_iter.py — __eq__ re-entrantly calls __setstate__ during next().
  • test_aiter_callable_sentinel_reentrant_exhaustion in test_asyncgen.py — __eq__ re-entrantly exhausts the iterator via a nested __anext__() send.

Verification (local, ASan, --without-pymalloc, arm64 macOS)

  • Both reporter PoCs: heap-use-after-free inside do_richcompare before the fix; clean run and correct semantics after (sync: next() returns the result; async: StopIteration with the produced value).
  • New regression tests pass; test_iter.py and test_asyncgen.py suites unchanged otherwise.

This PR was prepared with AI-agent assistance; the changes are small, reviewed line-by-line, and covered by the added regression tests.

@python-cla-bot

python-cla-bot Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

The following commit authors need to sign the Contributor License Agreement:

CLA not signed

…arison

The sentinel comparison in calliter_iternext() and
acallawaitable_handle_error() ran arbitrary Python code (a custom
__eq__) while holding only a borrowed reference to it_sentinel.
A re-entrant __setstate__ or __anext__ could release the sentinel
mid-comparison, leaving the comparison machinery using freed memory.
Hold a strong reference to the sentinel across PyObject_RichCompareBool.
@iliasabk
iliasabk force-pushed the fix-iter-sentinel-uaf branch from 610b155 to fbaa55b Compare September 25, 2026 00:04
@iliasabk

Copy link
Copy Markdown
Author

A note on scope, following picnixz's comment on #158033: since aiter is being reimplemented in pure Python, the async half may not matter for 3.16+. The two fixes in this PR are independent — I'm happy to split it so the iter() fix (#158031) lands on its own, and the aiter() half can then serve as the basis for the 3.13–3.15 backports once the approach there is decided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

uaf in aiter() uaf of it_sentinel in callable_iterator.__next__

1 participant