Conversation
|
The following commit authors need to sign the Contributor License Agreement: |
…arison The sentinel comparison in calliter_iternext() and acallawaitable_handle_error() ran arbitrary Python code (a custom __eq__) while holding only a borrowed reference to it_sentinel. A re-entrant __setstate__ or __anext__ could release the sentinel mid-comparison, leaving the comparison machinery using freed memory. Hold a strong reference to the sentinel across PyObject_RichCompareBool.
iliasabk
force-pushed
the
fix-iter-sentinel-uaf
branch
from
September 25, 2026 00:04
610b155 to
fbaa55b
Compare
Author
|
A note on scope, following picnixz's comment on #158033: since |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issues
Fixes #158031, fixes #158033 — two same-class heap-use-after-free reports in the callable-with-sentinel iterators added by gh-98469.
Root cause
Both
calliter_iternext()(synciter(callable, sentinel)) andacallawaitable_handle_error()(asyncaiter(callable, sentinel)) compareit->it_sentinelagainst the produced value withPyObject_RichCompareBool()while holding only a borrowed reference. The comparison can run arbitrary Python code via a custom__eq__, and that code can release the sentinel mid-comparison:Sentinel.__eq__callsit.__setstate__(((), StopIteration)), which doesPy_XSETREF(it->it_sentinel, NULL); once the__eq__frame is torn down the last reference is gone anddo_richcompare()continues with a freedv(ASan: UAF read atiterobject.c:285).Sentinel.__eq__drivesait.__anext__().__await__().send(None), which reachesacallawaitable_start→ callable raisesStopAsyncIteration→acalliter_exhaust()→Py_CLEAR(it->it_sentinel); same freed-vcontinuation (ASan: UAF read atiterobject.c:644).Fix
Hold a strong reference to the sentinel across the
PyObject_RichCompareBool()call in both functions — the same pattern already used elsewhere for callbacks that can mutate the iterator (see the ordering note incalliter_reduce, gh-101765). No behavior change: the in-flight comparison still uses the sentinel that was current when the comparison started, and subsequent state (calliter_exhaust,_PyGen_SetStopIterationValue) reads the post-mutation fields as before.Tests
test_calliter_sentinel_reentrant_setstateintest_iter.py—__eq__re-entrantly calls__setstate__duringnext().test_aiter_callable_sentinel_reentrant_exhaustionintest_asyncgen.py—__eq__re-entrantly exhausts the iterator via a nested__anext__()send.Verification (local, ASan,
--without-pymalloc, arm64 macOS)heap-use-after-freeinsidedo_richcomparebefore the fix; clean run and correct semantics after (sync:next()returns the result; async:StopIterationwith the produced value).test_iter.pyandtest_asyncgen.pysuites unchanged otherwise.This PR was prepared with AI-agent assistance; the changes are small, reviewed line-by-line, and covered by the added regression tests.