Repository navigation
Conversation
In the free-threaded build, type slot, MRO and __bases__ updates are prepared under TYPE_LOCK and published with the world stopped. The lock was "pinned" so that it stayed held while the thread waited for the stop-the-world mutex. That deadlocks against a thread that owns the stop-the-world mutex and needs TYPE_LOCK to re-attach, which happens when Python code (a custom mro(), a str subclass __eq__ of a class dict key, or an automatic GC) runs under the type lock and stops the world. Remove the pinning and publish optimistically instead. A type publish generation counter is incremented by every mutation that a prepared update might depend on. The preparing thread snapshots it, prepares, stops the world and compares. If the counter changed, nothing is published and the update is prepared again.
This is more clear than passing the "gen" as a pointer and using NULL to indicate that we don't want to check the generation.
Use explicit stop-the-world pauses and finite worker workloads. Replace slot-update stress with a deterministic reentrancy test, and consolidate queue reference-release coverage.
The rollback on error restores the old bases only if tp_bases is still the tuple this assignment stored, identified by object identity. That breaks if another assignment stores the same tuple object in between. Store a private copy so that the identity check is reliable.
Required since a dropped reference could run type_dealloc(), which takes TYPE_LOCK. Move the final slot_update_clear() after the version tags are invalidated.
Both helpers now return 1 when stored, 0 when the expected value was replaced and -1 when the publish generation changed. The check_gen flag is replaced by the TYPES_PUBLISH_GEN_NONE sentinel.
The checks are plain C and do not run Python code.
nascheme
force-pushed
the
gh-158195-unpin-type-lock
branch
from
October 21, 2026 06:03
6a77bee to
7917d6a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the free-threaded deadlock described in gh-158195 by removing type-lock pinning, rather than changing how stop-the-world restores critical sections. With this PR, the type lock and class dict lock can now be released normally while a thread waits to stop the world.
Design
Type slot, MRO and
__bases__updates are prepared underTYPE_LOCKand published with the world stopped. Without pinning, another thread can change the state between preparation and publication. Preparation itself can also release the lock or run reentrant Python code, so pinning did not fully protect against stale updates even before this change.Publication now uses an optimistic retry:
The counter advances on slot-relevant class dict mutations, MRO and bases changes, and slot publication. Ordinary class attribute assignments do not advance it. A single counter keeps validation simple, at the cost of possible retries for unrelated type mutations. Existing type version tags cannot serve this purpose: they may already be invalid (zero) throughout preparation.
A stale update must be recomputed, not simply discarded: an unrelated mutation can invalidate the snapshot without doing the work needed to bring this type's slots or MRO up to date. Reentrancy and rollback checks still avoid overwriting an MRO or bases value that has already been replaced.
Supporting changes
type_dealloc(), which takes the type lock again.__bases__validation is repeated on retry; argument checks and the audit event run only once per assignment.__bases__assignment stores a private copy of the assigned tuple, so the rollback on error can tell by identity whether tp_bases was replaced again, even if another assignment stored the same tuple object.The generation counter and retry checks are free-threaded-only; the GIL build has no generation-tracking overhead.
Backporting
This fix should be backported to 3.14 and 3.15, since the deadlock exists in both. For ABI compatibility, the generation counter could become a process global, rather than inside the interpreter struct.
The change is structured so that the default (GIL) build is essentially unaffected:
publish_genand thetypes_publish_*helpers exist only underPy_GIL_DISABLED. On the default build the macros expand to constants (genis 0,staleis 0), so no retry happens.try_set_tp_mro()/try_set_tp_bases()reduce on the default build to the sametp_mro == old_mroandtp_bases == new_basescompares the old code made at the same points (reentinmro_internal(), the undo loop, the bail path).slot_update_*,apply_type_slot_updates(), the loops inupdate_slot_after_setattr()andupdate_all_slots()) was already underPy_GIL_DISABLED._PyType_Modified_Unlocked()after the flag update intype_set_abstractmethods()and_PyType_SetFlagsRecursive()has no observable effect on the default build: nothing runs in between.BEGIN_TYPE_LOCK()has no effect there either; the lock is a no-op.__bases__assignment now stores a copy of the assigned tuple on both builds. The only observable difference is thatC.__bases__is no longer the same object that was assigned.mro_hierarchy_for_complete_type()is now conditional on tp_mro still being the MRO just set. This check is needed on the free-threaded build because another thread may replace the MRO while the type lock is released. On the default build, there is no intervening opportunity to run Python code. The conditional rollback therefore has no expected behavioral effect there.