Skip to content

gh-159603: Use pthread_attr_get_np() on FreeBSD - #159604

Merged
vstinner merged 3 commits into
python:mainfrom
vstinner:freebsd_stack
Oct 20, 2026
Merged

gh-159603: Use pthread_attr_get_np() on FreeBSD#159604
vstinner merged 3 commits into
python:mainfrom
vstinner:freebsd_stack

Conversation

@vstinner

@vstinner vstinner commented Oct 17, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

Use pthread_attr_get_np() on FreeBSD to get the stack size (and guard size) of the current thread.

Check if pthread_attr_get_np() function and <pthread_np.h> header are available in configure.

@vstinner
pythongh-159603: Use pthread_attr_get_np() on FreeBSD
e61d02d 
Use pthread_attr_get_np() on FreeBSD to get the stack size (and guard
size) of the current thread.

Check if pthread_attr_get_np() function and <pthread_np.h> header are
available in configure.
@bedevere-app bedevere-app Bot mentioned this pull request Oct 17, 2026
Closed
This was referenced Oct 17, 2026
Open
Open
@vstinner

vstinner commented Oct 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Sadly, I added the new HAVE_PTHREAD_NP_H macro to Tools/build/check_capi_macros_ignored.txt. I don't know how to check if a header is available in configure and define a macro name different than HAVE_<header>_H.

I asked on #157696 (comment) if there is a way to define a _Py_HAVE_xxx_H macro instead.

@vstinner

vstinner commented Oct 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Without this change, test_lazy_import.test_deeply_dotted_name() does crash with SIGILL on FreeBSD, see: #159535.

With this change, the test pass as expected (I tested FreeBSD 16):

$ ./python -m test test_lazy_import -v -m test_deeply_dotted_name
== CPython 3.16.0a1+dev (heads/pr/159604:c031a553097, Oct 17 2026, 02:48:37) [Clang 21.1.8 (https://github.com/llvm/llvm-project.git llvmorg-21.1.8-0-g2078da
== FreeBSD-16.0-CURRENT-amd64-64bit-ELF little-endian
== Python build: debug
(...)
test_deeply_dotted_name (test.test_lazy_import.LazyImportTypeTests.test_deeply_dotted_name)
A long dotted name must not overflow the C stack. ... ok
(...)
Result: SUCCESS

@vstinner vstinner mentioned this pull request Oct 17, 2026
Merged
@vstinner

vstinner commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

!buildbot FreeBSD16

@bedevere-bot

bedevere-bot commented Oct 17, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @vstinner for commit c031a55 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F159604%2Fmerge

The command will test the builders whose names match following regular expression: FreeBSD16

The builders matched are:

  • AMD64 FreeBSD16 Non-Debug PR
  • AMD64 FreeBSD16 PR

@vstinner

vstinner commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

Tail calling interpreter / x86_64-apple-darwin/clang (pull_request): Cancelled after 60m

I created #159612 for this unrelated failure.

@vstinner

vstinner commented Oct 18, 2026

Copy link
Copy Markdown
Member Author

test_json fails on FreeBSD 16, the failure seems to be related:

FAIL: test_highly_nested_objects_encoding (test.test_json.test_recursion.TestCRecursion.test_highly_nested_objects_encoding)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/pull_request.opsec-fbsd16/build/Lib/test/test_json/test_recursion.py", line 98, in test_highly_nested_objects_encoding
    with self.assertRaises(RecursionError):
         ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
AssertionError: RecursionError not raised

@vstinner
Update test_json.test_recursion
120fce0 
On FreeBSD 16 with a default stack size of 512 MB, the json C
implementation no longer raises RecursionError in
test_highly_nested_objects_encoding().

Reduce max_depth from 5_000 to 2_000 so
test_highly_nested_objects_encoding() of the Python implementation
only takes 1.6 seconds, instead of 12 seconds.
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Oct 19, 2026
@vstinner vstinner removed the type-feature A feature request or enhancement label Oct 19, 2026
@vstinner

vstinner commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

!buildbot FreeBSD

@bedevere-bot

bedevere-bot commented Oct 19, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 120fce0 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F159604%2Fmerge

The command will test the builders whose names match following regular expression: FreeBSD

The builders matched are:

  • AMD64 FreeBSD15 NoGIL PR
  • AMD64 FreeBSD15 PR
  • AMD64 FreeBSD14 PR
  • AMD64 FreeBSD Refleaks PR
  • AMD64 FreeBSD16 PR
  • AMD64 FreeBSD16 Non-Debug PR

corona10
corona10 reviewed Oct 19, 2026
View reviewed changes
Comment thread Python/ceval.c
void *stack_addr;
pthread_attr_t attr;
#ifdef _Py_HAVE_PTHREAD_ATTR_GET_NP
int err = pthread_attr_init(&attr);

@corona10 corona10 Oct 19, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already called here; is it safe to call here again?

@vstinner vstinner Oct 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pthread_attr_init() should be called every time you allocate a new pthread_attr_t attr on the stack memory: https://www.man7.org/linux/man-pages/man3/pthread_attr_init.3.html. Here the strange part is that the pthread_getattr_np() code path doesn't call pthread_attr_init(), but it calls pthread_attr_destroy().

At least, I can say that calling pthread_getattr_np() on a fresh pthread_attr_t attr variable without calling pthread_attr_init() fails on FreeBSD. So I had to add pthread_attr_init() call.

@vstinner vstinner Oct 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote a follow-up PR #159857 to also call pthread_attr_init() on Linux.

@vstinner

vstinner commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

buildbot/AMD64 FreeBSD Refleaks PR — Build done.
buildbot/AMD64 FreeBSD14 PR — Build done.
buildbot/AMD64 FreeBSD15 NoGIL PR — Build done.
buildbot/AMD64 FreeBSD15 PR — Build done.
buildbot/AMD64 FreeBSD16 Non-Debug PR — Build done.
buildbot/AMD64 FreeBSD16 PR — Build done.

Good: the test suite passed on the 6 FreeBSD buildbots.

@vstinner
vstinner merged commit 71ba39d into python:main Oct 20, 2026
69 of 72 checks passed
@vstinner
vstinner deleted the freebsd_stack branch October 20, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@corona10 corona10 corona10 left review comments
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping emmatyping is a code owner
@itamaro itamaro Awaiting requested review from itamaro itamaro is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vstinner @bedevere-bot @corona10