Skip to content

[3.15] gh-151518: Avoid STW starvation of attaching threads (GH-152826) - #158186

Open
tpn wants to merge 1 commit into
python:3.15from
tpn:codex/gh-151518-backport-315
Open

tpn wants to merge 1 commit into
python:3.15from
tpn:codex/gh-151518-backport-315

Conversation

@tpn

@tpn tpn commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Backport of gh-152826, merged as f156510, for gh-151518.

A tight loop of stop-the-world pauses can repeatedly suspend a thread trying to reattach after sleeping, preventing it from returning and stopping the loop. This was found during intentional numba-cuda GC stress on many-core machines. The fix preserves active attach waiters across successive pauses so they have a chance to run.

This branch predates gh-157839, so the waiter-aware resume operation is a private helper in pystate.c. The attachment changes, regression test, and guard that avoids blocking waits on WASI are carried over from the merged fix.

Validation:

  • Free-threaded debug and TSan: all 31 focused threading, GC, and asyncio tests passed.
  • With only the test changes, the unfixed branch hits the 30-second watchdog with 4 and 64 CPUs. The backport passes three runs each with 1, 4, and 64 CPUs.
  • Ordinary GIL and WASI builds pass the internal C API helper tests, including test_stop_the_world. Local WASI checks used SDK 34 and Wasmtime 48; the WASI CI job also passed with the branch's configured toolchain.

CI note: the required Tests workflow passed. The Intel macOS JIT debug job exceeded its one-hour limit; its log is unavailable, so the cause is not yet known. The other 18 JIT jobs passed.

@hugovk Given the 3.15 RC freeze, should this go into 3.15.0 or wait for the first maintenance release?

…thonGH-152826)

Preserve active attach waiters across successive stop-the-world pauses
so a tight collection loop cannot keep them from making progress.

Keep the waiter-aware resume operation local to pystate.c because this
branch does not have the newer BRC suspend and resume helpers.

(cherry picked from commit f156510)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant