Skip to content

fix(python): a parameterized base links its class, like class Leaf(Base[str]) - #3101

Merged
colbymchenry merged 1 commit into
mainfrom
claude/pr-2998-python-generic-bases
Oct 30, 2026
Merged

colbymchenry merged 1 commit into
mainfrom
claude/pr-2998-python-generic-bases

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Summary

A Python class with a parameterized base, like class Leaf(Base[str]), class Repo(models.Base[int]) or class Both(Mixin, Base[str]), is now linked to that base, in extraction and in the kernel alike. A call through a module global holding such a class, like current = Leaf() then current.run(), now reaches the method the base declares.

Lands #2998 by @JosefAschauer.

Root cause

The grammar parses Base[str] in a base list as a subscript, and the base-list reader took only bare identifiers and attributes, so the base was skipped. The class had no extends edge, and everything that follows the class hierarchy missed it. The resolver's own text reader, pythonBases, also looked up Base[str] verbatim and split Pair[str, int] at its inner comma.

What came from the contributor, and what we changed

From #2998, unchanged: the subscript unwrap in src/extraction/tree-sitter.ts and in the kernel's python.rs, the bracket drop in pythonBases, and the test file.

Ours:

  • The test's kernel arm now fails instead of skipping when a build expects a kernel (CODEGRAPH_KERNEL_EXPECT=1), as the other kernel-gated tests do.
  • The CHANGELOG bullet moved next to the other Python bullets, with the credit added.

Validation

Stage A: every changed line read, including the Rust. Scanner clean: the only hits are the test's temp-dir writes, its CODEGRAPH_KERNEL* env toggles, which it restores, and its ../src import. No binaries; the kernel was built here from python.rs.

Reproduced on main: class Leaf(Base[str]) has no extends edge, on the wasm path and in the kernel.

Tests: the test file runs on both arms. Red on main's source and kernel (12 of 16 fail, 6 per arm), green here (16 passed).

Kernel parity with this kernel, on 3,155 Python files of flask, requests, mypy, litestar, sqlalchemy, polar and django: 0 diffs, and the same 5 files defer to wasm as on main. The kernel and wasm edge dumps of flask, litestar, mypy and sqlalchemy are identical.

A/B against main, kernel on, every arm built from its own sources:

Repo Edges Added Removed Target moved Same target, new confidence
psf/requests @ 3c7baaa 2,976 → 2,976 0 0 0 0
pallets/flask @ 49943b5 5,287 → 5,292 5 0 0 12
litestar-org/litestar @ 716f8b0 42,289 → 42,397 108 0 12 43
python/mypy @ d8a43fb 70,754 → 70,811 57 0 0 0
sqlalchemy/sqlalchemy @ 2d8e649 228,406 → 229,431 1,025 0 82 159
polarsource/polar server/polar @ 8dd9956 62,008 → 62,305 297 0 513 9
django/django django/ @ a6a5d1e 46,228 → 46,228 0 0 0 0
expressjs/express @ 1c6f1b0 (control) 2,296 → 2,296 0 0 0 0
gin-gonic/gin @ 0f09c3a (control) 8,073 → 8,073 0 0 0 0

Both controls have byte-identical full dumps. polar and django are their package folders, to stay under the heavy-index gate; whole, they index 2,103 and 3,027 files.

Every edge class in the 7 Python repos, 457,948 edges on main:

Change Edges Class Read as
Added 610 extends by import Right
Added 291 extends by name 288 in the same file, right. The 3 across files are wrong (see below)
Added 555 calls at 0.9 Right in a sample of 20: t.insert() reaches TableClause.insert
Added 36 references at 0.9 Same walk up the hierarchy, right in the buckets read
Moved 492 calls 0.4 → 0.9 491 in polar: self.get_one_or_none() moves from the protocol stub RepositoryProtocol to RepositoryBase and its mixins
Moved 41 calls 0.7 → 0.9 litestar's GenericSyncMockRepository moves from AbstractAsyncRepository to AbstractSyncRepository, and sqlalchemy's Insert.ext from Query.ext to HasSyntaxExtensions.ext
Moved 47 calls on a receiver, 0.65 or 0.8 → 0.9 t.select() moves from FunctionElement.select to FromClause.select, and polar's repo.get_by_id() from another repository to the mixin
Moved 12 calls 0.9 → 0.4 or 0.7 Less right: sqlalchemy's self.op() moves from Operators.op to the typing-only overload SQLCoreOperations.op
Moved 15 The rest Right, except one Enum.copy that moves to TypeEngine.copy, a base further up the method resolution order
Same target 223 calls, higher confidence Right: the hierarchy now proves what was a guess
Removed 0

We read 44 sampled changes and every bucket under 50.

Cost, with #3001 on top: sqlalchemy, median of 3 interleaved runs: 23.7 s on main, 25.2 s here, with 1,143 more edges.

Suite: 832 files (826 passed, 6 skipped), 9,803 tests passed, 0 failed, 87 skipped, on this exact commit, with this kernel staged, 4 workers.

Not in this PR

  • A base the file imports from outside the project still links by name to a same-named project class. This is not new: on main, sqlalchemy's 28 class X(Enum) with from enum import Enum link to its own sql/sqltypes.py Enum, and litestar's 21 pydantic BaseModel classes link to a docs example. This PR adds 3 more of that kind in sqlalchemy, because Sequence[_T], Tuple[int, ...] and MutableSet[_T] from typing now reach the name match like their plain forms do. None of the three brought a follow-on call edge. The fix belongs in its own PR, for plain and parameterized bases together.
  • fix(python): a base list wrapped over several lines is read whole #3001 is stacked on this and lands next.

🤖 Generated with Claude Code

…se[str])

A Python class with a parameterized base had no extends edge. The grammar
parses Base[str] in a base list as a subscript, and the base-list reader took
only bare identifiers and attributes, so the base was skipped, on the wasm path
and in the kernel alike. Everything that follows the class hierarchy missed
such a class. The resolver's own text reader of a base list, pythonBases, also
looked up Base[str] verbatim and split Pair[str, int] at its inner comma, so
current = Leaf() then current.run() did not reach Base.run.

A subscript base is now unwrapped to its value before the identifier or
attribute check, in src/extraction/tree-sitter.ts and in the kernel's
python.rs. pythonBases drops bracketed type arguments before splitting the list.
Generic[T] and Protocol[T] stay outside the project like their plain forms; a
computed base, a subscript of a subscript and keyword arguments link nothing.

Validation:
- The test file runs on both arms: red on main's source and kernel (12 of 16
  fail, 6 per arm), green here (16 passed).
- Kernel parity with this kernel on 3,155 Python files of flask, requests,
  mypy, litestar, sqlalchemy, polar and django: 0 diffs, and the same 5 files
  defer to wasm as on main. The kernel and wasm edge dumps of the repos run
  both ways are identical.
- A/B against main on 7 Python repos, 457,948 edges: 1,492 added (901 extends,
  555 calls, 36 references), 0 removed, 607 call sites moved, 223 kept their
  target with higher confidence.
  polar: 491 self.get_one_or_none() style calls move from the protocol stub
  RepositoryProtocol at 0.4 to RepositoryBase and its mixin at 0.9.
  litestar: 12 calls in GenericSyncMockRepository move from
  AbstractAsyncRepository to AbstractSyncRepository.
  sqlalchemy: 435 extends and 554 calls added, like t.insert() reaching
  TableClause.insert.
  We read 44 sampled changes and every small bucket. They read right, except
  12 self.op() sites that move to the typing-only overload SQLCoreOperations.op
  and one Enum.copy that moves to TypeEngine.copy.
- 3 of the 901 extends are wrong: sqlalchemy's Sequence[_T], Tuple[int, ...]
  and MutableSet[_T] from typing link to its own classes of those names. The
  plain form does this on main (28 class X(Enum) there), and none of the 3
  brought a call edge.
- Controls express and gin: full dumps byte-identical.
- Index cost with PR #3001 on top, sqlalchemy, median of 3 interleaved: 23.7 s
  on main, 25.2 s here, with 1,143 more edges.

Original PR #2998 by @JosefAschauer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants