Repository navigation
fix(python): a base list wrapped over several lines is read whole - #3001
JosefAschauer wants to merge 3 commits into
Conversation
|
Thanks! The module-global case is real on main, your reader fixes it, and the tests pass on the newest main. One more reader has the same gap: the Python branch of |
declaredSupertypes, behind a call through the class name like Wrapped.run(x), read a Python base list from the first three lines and stopped at the first closing parenthesis, so a list wrapped past them (or holding a comment with a parenthesis) still had no bases there. It now reads the list through pythonBaseList, as pythonBases does. Review note from the maintainer on colbymchenry#3001. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq
|
Thanks, good catch. The test adds class-name calls (
Without the change, the wrapped, commented and decorated-wrapped cases fail; with it, all 8 pass. On the branch:
This branch is still stacked on #2998, so it carries that commit too. Marking both ready. |
|
Heads-up on ordering: this follow-up now changes the same case 'python': {
let bases = pythonBaseList(cls, context);
while (bases && /\[[^[\]]*\]/.test(bases)) bases = bases.replace(/\[[^[\]]*\]/g, '');
return names(bases?.replace(/\w+\s*=\s*[\w.]+/g, '').replace(/([A-Za-z_][\w.]*)\.\w+\s*\(/g, '$1 ('));
}With both merged, the full suite passes apart from the known failures: |
…se[str]) A Python class with a parameterized base had no extends edge. The grammar parses Base[str] in a base list as a subscript, and the base-list reader took only bare identifiers and attributes, so the base was skipped, on the wasm path and in the kernel alike. Everything that follows the class hierarchy missed such a class. The resolver's own text reader of a base list, pythonBases, also looked up Base[str] verbatim and split Pair[str, int] at its inner comma, so current = Leaf() then current.run() did not reach Base.run. A subscript base is now unwrapped to its value before the identifier or attribute check, in src/extraction/tree-sitter.ts and in the kernel's python.rs. pythonBases drops bracketed type arguments before splitting the list. Generic[T] and Protocol[T] stay outside the project like their plain forms; a computed base, a subscript of a subscript and keyword arguments link nothing. Validation: - The test file runs on both arms: red on main's source and kernel (12 of 16 fail, 6 per arm), green here (16 passed). - Kernel parity with this kernel on 3,155 Python files of flask, requests, mypy, litestar, sqlalchemy, polar and django: 0 diffs, and the same 5 files defer to wasm as on main. The kernel and wasm edge dumps of the repos run both ways are identical. - A/B against main on 7 Python repos, 457,948 edges: 1,492 added (901 extends, 555 calls, 36 references), 0 removed, 607 call sites moved, 223 kept their target with higher confidence. polar: 491 self.get_one_or_none() style calls move from the protocol stub RepositoryProtocol at 0.4 to RepositoryBase and its mixin at 0.9. litestar: 12 calls in GenericSyncMockRepository move from AbstractAsyncRepository to AbstractSyncRepository. sqlalchemy: 435 extends and 554 calls added, like t.insert() reaching TableClause.insert. We read 44 sampled changes and every small bucket. They read right, except 12 self.op() sites that move to the typing-only overload SQLCoreOperations.op and one Enum.copy that moves to TypeEngine.copy. - 3 of the 901 extends are wrong: sqlalchemy's Sequence[_T], Tuple[int, ...] and MutableSet[_T] from typing link to its own classes of those names. The plain form does this on main (28 class X(Enum) there), and none of the 3 brought a call edge. - Controls express and gin: full dumps byte-identical. - Index cost with PR #3001 on top, sqlalchemy, median of 3 interleaved: 23.7 s on main, 25.2 s here, with 1,143 more edges. Original PR #2998 by @JosefAschauer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pythonBases (behind module-global and self. dispatch) read only the `class` line, so a base list black or ruff wraps one base per line looked empty and `current = Wrapped(); current.run()` reached no base's method. A new pythonBaseList reads from the `class` line to the list's closing parenthesis, within the class, leaving out comments. Extraction already read the whole list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq
declaredSupertypes, behind a call through the class name like Wrapped.run(x), read a Python base list from the first three lines and stopped at the first closing parenthesis, so a list wrapped past them (or holding a comment with a parenthesis) still had no bases there. It now reads the list through pythonBaseList, as pythonBases does. Review note from the maintainer on colbymchenry#3001. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq
17e2f5b to
00819ff
Compare
|
Thanks for landing #2998, #3000 and #2999! I've rebased this one onto current main (08b187a). It now carries only its own three commits, and the |
Both of the resolver's readers of a Python base list stopped early. pythonBases, behind module-global and self. dispatch, read only the class line. declaredSupertypes, behind class-name calls, read three lines and needed the closing parenthesis on them. black and ruff wrap a long list one base per line, so class Wrapped(Mixin, Base) written that way seemed to have no bases: current = Wrapped() then current.run(), and Wrapped.run(x), reached nothing a base declares. Extraction already read the whole list. A new pythonBaseList reads from the class line to the list's closing parenthesis, counting nested parentheses, within the class's extent, and leaves out comments, so a ) in one doesn't end the list. Both readers use it. declaredSupertypes still drops type arguments first, the composition the author posted on PR #3001, and keeps its metaclass= and factory rewrites. Validation: - The test file: 7 of 9 fail without this change (the two one-line controls pass), 9 pass here. Putting the type-argument drop out of the merged case 'python' fails the new composition case and the two of PR #3083. - Wrapped.run(None) on a wrapped base list reaches Base::run with the kernel and with wasm, as do a list over five lines, a decorated class, and a wrapped Base[Optional[User]]. - A/B on top of PR #2998, 7 Python repos, 459,440 edges: 118 added (102 calls, 16 references), 0 removed, 959 call sites moved, 113 kept their target with higher confidence. requests, flask, litestar, mypy and django do not change. polar: 942 XRepository.from_session(session) calls move from a sibling repository's from_session at 0.65 to RepositoryBase.from_session at 0.85. sqlalchemy: wrapped heads like class Table( and class ColumnElement( now reach what they inherit: t.select() reaches FromClause.select. Its 17 moved sites read right; 15 of them land on the typing-only SQLCoreOperations overload of the same ColumnOperators method. - Controls express and gin: full dumps byte-identical. - No kernel change. Index cost with PR #2998, sqlalchemy, median of 3 interleaved: 23.7 s on main, 25.2 s here. Original PR #3001 by @JosefAschauer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks @JosefAschauer! This landed as #3103 with you as the commit author, using the combined resolution you posted for |
Stacked on #2998. This branch contains #2998's commit (89624ed). The change here is the second commit, 3b6b3f8. I'll rebase it once #2998 lands.
Problem
The resolver's own reader of a class's bases (
pythonBases, behind module-global andself.dispatch) reads only theclassline. black and ruff wrap a long base list one base per line:Such a class looked like it had no bases. Extraction already reads the whole list, so the class has its
extendsedges; only this reader missed them.Fix
A new
pythonBaseListreads from theclassline to the list's closing parenthesis, counting nested parentheses, within the class's extent. It leaves out#comments, so a)in a comment doesn't end the list.pythonBasesuses it in place of the single-line regex.Validation
New
__tests__/python-multiline-bases.test.ts: module globals over a one-line list (control), a wrapped list, a wrapped list with comments (one ends in:)), and a parameterized base wrapped inside the list. Each resolvesruntoBase::run, with a decoyOther.runin the file.Mutation-checked twice: reading only the first line fails the three wrapped cases, and leaving comments in fails the comment case.
npx tsc --noEmit: clean.Full suite on Linux: 8772 passed, 4 failed. These are the same environmental failures as fix(python): a parameterized base links its class, like class Leaf(Base[str]) #2998 (
bundle-launcher×2,cli-sync×2).Independent review (Grok): SHIP.
🤖 Generated with Claude Code
https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq