Skip to content

Pytest discovery: create_class_node re-parses the file for every class on Python < 3.13 #26240

Open

Description

@fyulingi

Type: Performance Issue

Environment

  • Python extension: ms-python.python 2026.10.0
  • VS Code: 1.134.0
  • OS: Linux (Remote - SSH)
  • Python: 3.10.22
  • pytest: 9.1.1

Summary

To put a line number on class nodes, vscode_pytest.create_class_node() calls
inspect.getsourcelines(class_module.obj) once per test class. On Python < 3.13, classes have no
__firstlineno__, so inspect.findsource() re-parses the entire module with ast.parse()
and walks the whole AST on every call. A file containing N test classes is therefore parsed N times.

Test discovery that takes ~4.6s with plain pytest takes ~45s in VS Code. During that time the
Testing view is empty and no gutter run buttons are shown.

Repro

Generate 20 files × 50 classes × 20 tests (20,000 tests, 1,000 classes):

# gen.py
import pathlib

root = pathlib.Path("tests")
root.mkdir(exist_ok=True)
for f in range(20):
    lines = []
    for c in range(50):
        lines.append(f"class TestGroup{c}:")
        for m in range(20):
            lines.append(f"    def test_case_{m}(self):\n        assert {m} == {m}\n")
    (root / f"test_file_{f}.py").write_text("\n".join(lines))

Run discovery with and without the extension's plugin (P = the extension's python_files dir):

python gen.py
python -m pytest --collect-only -q -p no:cacheprovider tests

TEST_RUN_PIPE=/tmp/out.json PYTHONPATH=$P:$P/lib/python \
  python -m pytest -p vscode_pytest --collect-only -q -p no:cacheprovider tests

Results (warm runs, same machine)

Python 3.10.22 Python 3.13.16
plain pytest --collect-only 4.6s 2.9s
with -p vscode_pytest 45.1s 4.8s
with -p vscode_pytest + per-file cache (see below) 8.2s n/a

Profile (Python 3.10, repro above)

ncalls  cumtime  function
     1   56.521  vscode_pytest/__init__.py:698(build_test_tree)
  1000   54.293  vscode_pytest/__init__.py:877(create_class_node)
  1000   54.263  inspect.py:1112(getsourcelines)
  1000   51.794  inspect.py:932(findsource)
  1000   30.349  ast.py:33(parse)
  1067   30.345  {built-in method builtins.compile}

Nearly all of discovery time goes to findsource → ast.parse for classes. Times are inflated by cProfile overhead (45s wall-clock without profiling); the proportions are what matter.

Root cause

python_files/vscode_pytest/__init__.py:

def create_class_node(class_module):
    ...
    _, lineno = inspect.getsourcelines(class_module.obj)
  • Only lineno is needed. getsourcelines also extracts the full source block, which is extra work.
  • On CPython < 3.13, inspect.findsource() for a class runs ast.parse(source) plus a
    _ClassFinder visit of the whole file on every call. Nothing caches it, so the cost is
    O(classes × file size).
  • On 3.13+ it reads cls.__firstlineno__, which is why 3.13 is fine.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions