Repository navigation
Conversation
Programs/_freeze_module and _bootstrap_python were linked with
$(LIBRARY_OBJS_OMIT_FROZEN), i.e. every static module from all Setup
files (Setup.local, Setup.stdlib with MODULE_BUILDTYPE=static, ...).
That makes it impossible for a builtin module to depend on anything the
helpers produce: a frozen module header, Modules/getpath.o, or generated
sources. A Setup.local module including a header generated by
$(FREEZE_MODULE) is a dependency cycle ("Circular ... dependency
dropped"), and one referencing a getpath.o symbol fails to link, since
_freeze_module links Modules/getpath_noop.o instead. CPython itself
needed a workaround for the latter: a stub of
_Py_Get_Getpath_CodeObject() in getpath_noop.c for _testinternalcapi
in MODULE_BUILDTYPE=static builds.
Mirror what PCbuild/_freeze_module.vcxproj already does on Windows
(a fixed source list plus PC/config_minimal.c): link the helpers with
the core objects plus the modules of Modules/Setup.bootstrap only.
* makesetup: new "-b Setup" option marking a Setup file as the bootstrap
modules. Their objects and libraries are substituted for
_BOOTSTRAP_MODOBJS_ / _BOOTSTRAP_MODLIBS_, and a second table,
config_bootstrap.c, is generated with only those modules. A bootstrap
module overridden by an earlier rule (Setup.local) keeps the
overriding static module for the helpers too.
* Makefile.pre.in: BOOTSTRAP_LIBRARY_OBJS (core + Modules/config_bootstrap.o
+ $(BOOTSTRAP_MODOBJS)) replaces LIBRARY_OBJS_OMIT_FROZEN in the
_freeze_module and _bootstrap_python links. getbuildinfo gets a
separate object for the helpers, as its prerequisites include
$(MODOBJS).
* configure.ac / configure: pass -b Modules/Setup.bootstrap and move
config_bootstrap.c into Modules/ (configure hand-edited in sync).
* Modules/getpath_noop.c: drop the now unnecessary stub.
The final libpython/python link is unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YD8kPnXaAMJZk2LvSHPXYA
Setup.local builtin modules that depend on build-helper output; see Tools/build/bootstrap-split-repro/README.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD8kPnXaAMJZk2LvSHPXYA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Programs/_freeze_moduleand_bootstrap_pythonare linked with$(LIBRARY_OBJS_OMIT_FROZEN), i.e. with every static module from all Setup files (Setup.local, and all ofSetup.stdlibunderMODULE_BUILDTYPE=static). As a result a builtin module cannot depend on anything the helpers themselves produce:$(FREEZE_MODULE)is a dependency cycle (make: Circular ... dependency dropped, then a compile failure);Modules/getpath.ofails to link_freeze_module, which linksModules/getpath_noop.oinstead.CPython itself already carries a workaround for the second case: the
_Py_Get_Getpath_CodeObject()stub ingetpath_noop.c, there only because_testinternalcapigets linked into_freeze_moduleinMODULE_BUILDTYPE=staticbuilds. That option is not available to out-of-treeSetup.localmodules.The Windows build already does this right:
PCbuild/_freeze_module.vcxprojcompiles a fixed source list withPC/config_minimal.c.Change
Link the helpers with the core objects plus the modules of
Modules/Setup.bootstraponly. The finallibpython/pythonlink is unchanged.Modules/makesetup: new-b Setupoption marking a Setup file as the bootstrap modules. Their objects and libraries are substituted for_BOOTSTRAP_MODOBJS_/_BOOTSTRAP_MODLIBS_, and a second table,Modules/config_bootstrap.c, is generated with only those modules. A bootstrap module overridden by an earlier rule (e.g. inSetup.local) keeps the overriding static module for the helpers too.Makefile.pre.in:BOOTSTRAP_LIBRARY_OBJS(core +Modules/config_bootstrap.o+$(BOOTSTRAP_MODOBJS)) replacesLIBRARY_OBJS_OMIT_FROZENin the_freeze_moduleand_bootstrap_pythonlinks.getbuildinfogets a separate object for the helpers, since its prerequisites include$(MODOBJS).configure.ac/configure: pass-b Modules/Setup.bootstrapand moveconfig_bootstrap.cintoModules/.Modules/getpath_noop.c: drop the now unnecessary stub.Testing
Linux x86-64, gcc 13:
Setup.local: same 37 builtin / 75 shared as before;test_frozen test_embed test_sysconfig test_importlib test_zipimport test_site test_getpath test_capipass; a secondmakeis a no-op.Setup.localcontaining (a) a builtin whose source#includes a header generated by$(FREEZE_MODULE)and (b) a builtin calling_Py_Get_Getpath_CodeObject(): fails onmain(circular dependency / missing symbol), builds and works on this branch.nmconfirms neither module (nor_asyncio) is in the helpers, whileposixis.MODULE_BUILDTYPE=static: 105 builtins inpython;_testinternalcapiand_sslare no longer in_freeze_module(37 MB vs 63 MB forpython), which exercises the removed stub. Same with aSetup.localoverride of a bootstrap module (errno): the override reaches the helpers.test_frozen test_embed test_getpath test_sysconfig test_capi test_import test_errnopass.Notes
MODULE_DEPS_STATIC=Modules/config.cstill recompiles the bootstrap objects wheneverSetup.localchanges, which relinks the helpers and regenerates all frozen headers — the same cascade as today. A follow-up could emitMODULE_DEPS_BOOTSTRAP=Modules/config_bootstrap.cfor-bmodules, providedconfig_bootstrap.cis only rewritten when its content changes.configurewas edited in sync withconfigure.ac;make regen-configurestill to be run.