Skip to content

gh-159717: Avoid excessive backtracking in glob pattern translation - #159861

Closed
xgbtxy wants to merge 2 commits into
python:mainfrom
xgbtxy:fix/gh-159717-glob-backtracking
Closed

xgbtxy wants to merge 2 commits into
python:mainfrom
xgbtxy:fix/gh-159717-glob-backtracking

Conversation

@xgbtxy

@xgbtxy xgbtxy commented Oct 20, 2026 •

Copy link
Copy Markdown

Fixes #159717.

Thanks to @digit50 for the original report and reproducer.

glob.translate() currently discards the star positions returned by fnmatch._translate(). As a result, short patterns such as '*a' * 8 + '*b' repeatedly backtrack in PurePath.match(), PurePath.full_match() and Path.glob().

Reuse the atomic-star joining logic for each ordinary glob segment, with a separator-excluding star. Keep the existing fast path for segments with fewer than two stars, and leave recursive ** and hidden-name handling in glob. fnmatch.translate() retains its existing public regex output.

A fixed part that can consume a separator must remain backtrackable: for example, *[!x]*x currently matches a/x. The separator check also covers case-sensitive and case-insensitive uses with custom separators. This patch preserves that behavior instead of changing character-class semantics as a side effect.

Validation on Windows x64, a Debug build of main at 4c73f50ca9885e5850dae64463e87cf5e80992a1:

  • test_fnmatch, test_glob, test_pathlib, test_shutil, test_tarfile, test_zipfile, test_unittest: all seven modules passed; regrtest reported run=4,202 skipped=740.
  • After restoring the simple-segment fast path, the three directly affected modules passed again (run=1,471 skipped=533). Skips include platform/feature-dependent tests; this is not a Linux validation run.
  • A deterministic comparison with the original translator, including hidden paths, recursion, character classes, custom separators and case-insensitive matching, found no matching differences across the sampled cases.
  • Tools/patchcheck/patchcheck.py and git diff --check pass; a NEWS entry is included.

For the issue's pattern against 36 a characters, the same Debug interpreter measured full_match at about 631 ms before and 0.096 ms after, and a single-file Path.glob at about 1706 ms before and 0.342 ms after. These are warmed-up medians of three calls for this reproducer, not a general performance claim.

This does not promise to remove all backtracking involving recursive patterns or character classes that consume separators. Cold translation of multi-star patterns has some additional cost; the change targets excessive matching cost and retains the common zero/one-star translation path.

Prepared with OpenAI Codex assistance for implementation, test generation and review. Builds, tests and measurements were executed locally.

@xgbtxy
xgbtxy requested a review from barneygale as a code owner October 20, 2026 06:21
Copilot AI balanced review requested due to automatic review settings October 20, 2026 06:21
@python-cla-bot

python-cla-bot Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@StanFromIreland

Copy link
Copy Markdown
Member

Sorry but I'm closing, the issue author asked to open the PR, please read the issue before opening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pathlib match, full_match and glob take exponential time on short patterns because glob.translate skips fnmatch's atomic-group rewrite

3 participants