Repository navigation
Conversation
Resolve a frame or traceback from its registered globals owner only when its canonical filename matches. Reuse the existing filename cache and retain the fallback for other objects and namespaces. Developed and tested with Codex (AI assistance).
The fixture left a stale __file__ value beneath a module subclass property. LOAD_ATTR_MODULE specialization can read that value, making the assertion depend on warmup state in the existing filename fallback, including main. Remove the shadow value and exercise warmed getfile access, two changing origins, and independent cold frame/traceback lookups. Keep production code unchanged. Verified the corrected test 50 times on each of main and the candidate; all six focused suites pass (1,135 tests reported, four skipped).
|
@eendebakpt, would you be willing to review this follow-up to #92042? It implements the frame/traceback split you suggested, without the registry watchers or C changes, and is independent of and complementary to #156685. CI is green; regression coverage and reproducible benchmarks are linked in the description. I’d particularly appreciate your view on verified frame globals taking precedence for an uncached filename collision. Thanks for your earlier feedback. |
| except (TypeError, FileNotFoundError): | ||
| pass |
There was a problem hiding this comment.
| except (TypeError, FileNotFoundError): | |
| pass | |
| except FileNotFoundError, TypeError: | |
| pass | |
| return None |
| if (type(module) is not types.ModuleType or | ||
| module.__dict__ is not namespace or '__getattr__' in namespace or | ||
| type(namespace.get('__file__')) is not str): |
There was a problem hiding this comment.
| if (type(module) is not types.ModuleType or | |
| module.__dict__ is not namespace or '__getattr__' in namespace or | |
| type(namespace.get('__file__')) is not str): | |
| if (type(module) is not types.ModuleType | |
| or module.__dict__ is not namespace | |
| or '__getattr__' in namespace | |
| or type(namespace.get('__file__')) is not str): |
| frame = object if isframe(object) else ( | ||
| object.tb_frame if istraceback(object) else None) |
There was a problem hiding this comment.
| frame = object if isframe(object) else ( | |
| object.tb_frame if istraceback(object) else None) | |
| if isframe(object): | |
| frame = object | |
| elif istraceback(object): | |
| frame = object.tb_frame | |
| else: | |
| frame = None |
| module = sys.modules.get(name) | ||
| # Leave dynamic module metadata to the existing filename fallback. | ||
| if (type(module) is not types.ModuleType or | ||
| module.__dict__ is not namespace or '__getattr__' in namespace or |
There was a problem hiding this comment.
Is '__getattr__' in namespace ever needed?
| @property | ||
| def __file__(self): | ||
| return self.source | ||
| # Do not leave a shadow value that LOAD_ATTR_MODULE can read instead |
| @@ -0,0 +1,3 @@ | |||
| Speed up :func:`inspect.getmodule` for frames and tracebacks whose globals | |||
| belong to a registered module with a matching filename. The globals owner | |||
There was a problem hiding this comment.
This promises a bit more than the code does. For example:
import inspect, os, sys, tempfile, types
fn = os.path.join(tempfile.mkdtemp(), 'owner.py')
open(fn, 'w').close()
owner = types.ModuleType('owner_mod'); owner.__file__ = fn
other = types.ModuleType('other_mod'); other.__file__ = fn
sys.modules['owner_mod'] = owner
sys.modules['other_mod'] = other # inserted later -> wins a registry scan
exec(compile("import sys\nframe = sys._getframe()\n", fn, 'exec'), owner.__dict__)
frame = owner.__dict__['frame']
print("1 frame, cold :", inspect.getmodule(frame).__name__)
inspect.getmodule(compile("pass", "<nowhere>", "exec")) # unrelated miss
print("2 frame, after a miss :", inspect.getmodule(frame).__name__)
inspect.modulesbyfile.clear(); inspect._filesbymodname.clear()
print("3 frame, cold again :", inspect.getmodule(frame).__name__)
inspect.modulesbyfile.clear(); inspect._filesbymodname.clear()
print("4 code object, cold :", inspect.getmodule(frame.f_code).__name__)
inspect.modulesbyfile.clear(); inspect._filesbymodname.clear()
inspect.getmodule(frame)
print("5 code obj after frame :", inspect.getmodule(frame.f_code).__name__)
This is a pathological case (intentionally giving two modules the same filename), so maybe fine to keep the code as is. But rewrite the news entry.
There was a problem hiding this comment.
Also please indicate this change in the docs of inspect directly. This may be breaking for some people.
Documentation build overview
|
When
inspect.getmodule()encounters an uncached registered frame or traceback, it currently scanssys.modulesto populate its filename map. Resolve that case directly from the frame's globals owner when both the registered-module identity and canonical filename match, then reuse the existing filename cache.This is the frame-path split requested in #92042. It is independent of and complementary to #156685, which removes a redundant scan during key computation. Both changes compose cleanly in either order. This PR changes only the Python frame lookup path; it does not include registry watchers, C changes or negative caching.
The existing
getabsfile(module)canonicalizer preserves source/bytecode normalization. Private execution namespaces, mismatched origins, module subclasses and dynamically supplied filenames, fileless modules and unrelated filename overrides retain the filename fallback. Existing filename-cache hits retain precedence. On an uncached filename collision, the verified globals owner wins over an unrelated module using the same filename. A later unrelated miss can repopulate the filename map from the registry and change that result; code-object lookups also share this map. This lookup-history dependence is documented and explicitly tested.Performance
The original, source-pinned measurement matrix used one current-main CPython 3.16
-O3Linux build, 15 interleaved fresh-process pairs per registry size, with 10,000 synthetic modules added to the ordinary registry:The bytecode fix is a separate measurement variant, not part of this PR. The exact uncorrected #156685 also shows the independent cold registered-frame gain: 87.01 ms → 4.660 µs, paired speedup 19,427.8× (18,548.4–20,458.2).
These are workload-specific synthetic measurements. Cold tests reset inspect's filename dictionaries, not the interpreter or filesystem caches. The import-like test retains other warmed entries and includes inserting/removing the target registry entry and evicting its own filename metadata. Mixed stacks containing an unregistered generated frame still require a full scan and receive no comparable benefit. We claim no steady-state stack or moduleless-miss speedup.
Eight further fresh-process crossover runs with counterbalanced within-process ABBA/BAAB batches found no meaningful regression in the tested warm paths. For the exact current replacement versus both changes: warmed frame ratio 1.002× (0.991–1.010), warmed source-context stack ratio 1.006× (0.997–1.016). Identical-code A/A controls are included in the evidence; small cross-process changes are not treated as gains.
A separate before/after recheck of the review revisions uses eight fresh processes for cold workloads and 16 fresh processes for refined warm-path measurements, with counterbalanced within-process batches and identical-code controls. It finds no statistically resolved change: cold-stack ratio 1.006× (0.997–1.014), newly registered frame 0.998× (0.986–1.010), warmed frame 0.992× (0.981–1.004), warmed stack 0.999× (0.986–1.011). These intervals leave smaller effects unresolved; no additional gain is claimed. The existing cache-hit instruction prefix is identical. Scripts, snapshots, raw samples and validation logs.
Validation
__getattr__no-scan test fails on the parent for the expected scan and passes on this revision. 300 repeated dynamic-filename, dynamic-loader and module-subclass tests pass across parent and revision.sphinx-lint --enable=default-role) andgit diff --checkpass.Measurement scripts, complete raw process samples, intervals, controls, exact sources/hashes and logs: https://github.com/mdeck/cpython/tree/inspect-frame-fastpath-evidence-20261022/Tools/benchmarks/inspect_getmodule_92041.
Developed and tested with Codex (AI assistance).