Skip to content

Skip directory entries that raise PermissionError in scandir - #15243

Open
Danish-kh-IT wants to merge 3 commits into
pytest-dev:mainfrom
Danish-kh-IT:fix/9089-scandir-permission-error
Open

Danish-kh-IT wants to merge 3 commits into
pytest-dev:mainfrom
Danish-kh-IT:fix/9089-scandir-permission-error

Conversation

@Danish-kh-IT

Copy link
Copy Markdown

Fixes #9089.

scandir() in src/_pytest/pathlib.py probes each directory entry with entry.is_file() and re-raises any OSError whose errno is not in _IGNORED_ERRORS. EACCES was missing from that set, so a single entry that cannot be inspected because of insufficient permissions (for example a symlinked folder owned by root) crashed the whole collection with PermissionError, and neither norecursedirs nor --ignore could prevent it.

This adds EACCES to _IGNORED_ERRORS, so such entries are skipped, consistent with how a vanishing file (ENOENT) and the other transient errors are already handled. Other OSErrors still propagate — the existing test_scandir_handles_os_error guarantee is unchanged.

Tests: added test_scandir_skips_permission_denied_entry next to test_scandir_handles_os_error, using the same mock-based approach. Verified locally: testing/test_pathlib.py 113 passed, 1 skipped; testing/test_collection.py + testing/test_main.py 131 passed, 10 skipped, 1 xfailed.

As noted in my comment on #9089, if maintainers would prefer a warning instead of silently skipping, I'm happy to adjust.

@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Oct 22, 2026

@ege-arhan ege-arhan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against the branch (c76c5bfa6). The new test passes, but the reported scenario still crashes — the PermissionError in #9089 does not come from entry.is_file().

scandir() only guards the per-entry probe (src/_pytest/pathlib.py:1004-1011); os.scandir(path) at src/_pytest/pathlib.py:996-1000 is still wrapped in except FileNotFoundError only, so recursing into a directory the user cannot read raises before _IGNORED_ERRORS is ever consulted:

$ mkdir -p p/locked p/ok && echo "def test_a(): pass" > p/ok/test_ok.py && chmod 000 p/locked
$ pytest p
E   PermissionError: [Errno 13] Permission denied: '.../p/locked'
ERROR locked - PermissionError: ...

Same result for the symlink case from the issue (ln -s /root p/sym_to_root), and both tracebacks bottom out at pathlib.py:998, i.e. the os.scandir(path) call itself, not at entry.is_file(). visit() (pathlib.py:1026) reaches it as soon as entry.is_dir() is true, so norecursedirs/--ignore still cannot help — which is exactly what the report describes.

The unit test hides this because it patches os.scandir wholesale, so the failing call site is mocked out:

with unittest.mock.patch("os.scandir") as mock_scandir:
    mock_scandir.return_value.__enter__.return_value = [denied_entry, ok_entry]
    assert scandir("/fake/path") == [ok_entry]

A pytester/tmpdir test that really chmods a subdirectory would have caught it.

Suggested handling for the unreadable directory itself:

try:
    scandir_iter = os.scandir(path)
except OSError as err:
    if _ignore_error(err):
        return []
    raise

FileNotFoundError remains covered since ENOENT is in _IGNORED_ERRORS, and a directory that cannot be listed then behaves like a missing one instead of aborting the session. Worth keeping the per-entry guard as well — a file that vanishes between readdir and stat is a different, still-real case.

Minor: EACCES is appended in the middle of the tuple ((ENOENT, ENOTDIR, EACCES, EBADF, ELOOP)), which breaks the alphabetical order the original comment/first three entries follow; also the changelog says "consistent with other transient filesystem errors", but EACCES is a permission fact rather than a race, so the wording could name the user-facing outcome (unreadable directories are skipped) instead.

Danish-kh-IT and others added 2 commits October 23, 2026 11:12
Test collection crashed with PermissionError when a directory contained an entry whose is_file() raised it (for example a symlinked folder owned by root), because EACCES was not among the errors ignored while scanning a directory. Add EACCES to _IGNORED_ERRORS so such entries are skipped, consistent with ENOENT and the other transient errors; other OSErrors still propagate. Fixes pytest-dev#9089.

Co-authored-by: Muse <noreply@muse.dev>
- scandir(): guard the os.scandir(path) call itself with _ignore_error, so recursing into an unreadable directory returns [] instead of crashing collection with PermissionError (the reported pytest-dev#9089 scenario); the per-entry guard is kept
- _IGNORED_ERRORS reordered alphabetically
- changelog reworded to name the outcome (unreadable directories are skipped)
- tests: mock-based tests for the directory-level call (PermissionError returns [], a non-ignorable OSError still propagates) plus a pytester test that really chmods a subdirectory

Co-authored-by: Muse <noreply@muse.dev>
@Danish-kh-IT
Danish-kh-IT force-pushed the fix/9089-scandir-permission-error branch from c76c5bf to 0c52f40 Compare October 23, 2026 06:19
The new chmod-based regression test exposed a second wall behind scandir(): while collecting an unreadable directory, _loadconftestmodules probes for <dir>/conftest.py with Path.is_file(), which stats inside the directory and raises PermissionError, interrupting the whole session. Treat a PermissionError from the probe as 'no conftest' - a directory that cannot be searched cannot provide one.

Co-authored-by: Muse <noreply@muse.dev>
@Danish-kh-IT

Copy link
Copy Markdown
Author

Update — the real chmod test you asked for earned its keep. With the scandir() guard in place it still failed on Linux CI, and the traceback showed a second wall behind the first: while collecting the unreadable directory, _loadconftestmodules probes for

/conftest.py with Path.is_file(), which stats inside the directory and raises PermissionError, interrupting the whole session ("Interrupted: 1 error during collection"). That probe was unreachable before, because scandir() crashed first — including in your repro.

Fixed in 839aaf5: a PermissionError from the conftest probe is now treated as "no conftest" — a directory that cannot be searched cannot provide one. The same change also covers the earlier round: scandir() guards the os.scandir(path) call itself, _IGNORED_ERRORS is alphabetical again, the changelog names the outcome (unreadable directories are skipped), and three tests were added (two mock-based ones for the directory-level call, plus the pytester chmod test).

Local verification: testing/test_conftest.py + testing/test_config.py 365 passed, 5 skipped, 1 xfailed; testing/test_pathlib.py + test_collection.py + test_main.py 246 passed, 12 skipped, 1 xfailed; pre-commit all passed. The chmod test still skips on my Windows machine, so the Linux CI jobs are the real check for it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Permission Denied Error for root owned folder

2 participants