Repository navigation
[rush] Resolve pnpm catalog references in injected cross-subspace dependencies - #6148
Draft
Bharat Middha (bmiddha) wants to merge 2 commits into
Conversation
PNPM resolves "catalog:" only for its own workspace importers. A Rush project from another subspace is injected as an external "file:" package, so PNPM rejects its catalog references (or would use the wrong subspace's catalog). The subspace pnpmfile shim now rewrites catalog references in dependencies, optionalDependencies and peerDependencies of projects from other subspaces, using the effective catalogs of the subspace that contains each project. The catalogs are passed through new optional pnpmfile settings. The injected dependencies hash also hashes the resolved versions, so catalog edits trigger a reinstall. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Now that the subspace pnpmfile shim resolves `catalog:` references in injected cross-subspace Rush projects, migrate the 98 literal dependencies/optionalDependencies specs that #6146 kept in the build-tests-subspace injected closure (23 projects) to `catalog:`, and add the 42 corresponding entries to globalCatalogs.default. Resolved versions are unchanged: lockfile packages/snapshots are identical, and sha512 integrity hashes are preserved. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of stack #6134, above #6146.
Summary
Problem. A subspace can inject Rush projects from another subspace. The subspace pnpmfile shim rewrites those
workspace:dependencies tofile:<relative path>, so pnpm treats the injected projects as external packages and resolves theirdependenciesandoptionalDependenciesinside the consuming subspace's install. pnpm only resolvescatalog:for the projects of its own workspace, so acatalog:spec in an injected project fails:Even if pnpm could resolve it, it would use the consuming subspace's catalog, which is the wrong one.
In this repo,
build-tests-subspaceinjects 35 default-subspace projects, transitively. #6146 worked around the error by keeping 98 prod/optional specs in that closure literal. That left a trap: after #6141,rush addof a prod dependency into any of those projects writescatalog:, and thebuild-tests-subspaceinstall breaks.Fix. This PR has two commits.
[rush]: inreadPackage, the subspace pnpmfile shim now replacescatalog:andcatalog:<name>in injected cross-subspace Rush projects with the literal entry from the catalogs of the project's own subspace.[repo]: catalogs the 98 literal prod/optional specs in the injected closure, reversing the [rushstack] Use pnpm catalogs for external dependency versions #6146 workaround.Details
Commit 1:
[rush] Resolve pnpm catalogs in injected cross-subspace depsIPnpmfile.ts,SubspacePnpmfileConfiguration.ts):ISubspacePnpmfileShimSettingsgainscatalogsBySubspaceName?: Record<string, PnpmCatalogs>, andIWorkspaceProjectInfogainssubspaceName?. Both are optional and additive; the settings and the shim are always generated by the same Rush version.getEffectivePnpmCatalogsfor each other subspace that has an injected project: the subspace'spnpm-config.json, falling back tocommon/config/rush/pnpm-config.json([rush] Use the common pnpm-config.json fallback for every pnpm catalog consumer #6138).SubspaceGlobalPnpmfileShim.ts):readPackagecallsresolveOtherSubspaceCatalogReferences(pkg)before the existingworkspace:→file:rewrite and before the user pnpmfile.settings.workspaceProjects, which holds only Rush projects from other subspaces. The current subspace's own importers are never rewritten: pnpm resolves those natively, and their lockfile importer specifiers keepcatalog:.dependencies,optionalDependenciesandpeerDependencies, the fields pnpm resolves for an installed package.devDependenciesof an external package are not installed.catalog:andcatalog:<name>(catalog:defaultis the same ascatalog:), parsed with the helpers from [rush] Resolve PNPM catalog references in implicitly preferred versions #6139 (PnpmCatalogReference.ts).reset()export clears the shim's module state, for tests.api/Subspace.ts):getPackageJsonInjectedDependenciesHashnow resolves catalog refs of projects outside the subspace before hashing. Without this, editing a catalog entry used by an injected project would not invalidate the consuming subspace's install.file:packages and snapshots resolve to the same versions as the literals did, so the lockfile content is unchanged. This repo'sbuild-tests-subspacelockfile is byte-identical.Other Rush code that reads injected manifests. I checked each candidate for mis-handling of
catalog::PnpmShrinkwrapFile.isWorkspaceProjectModifiedAsync: it compares only the subspace's own importers, which pnpm resolves natively and which still containcatalog:. No change needed.BaseInstallManager, pnpm >= 12 injected path ([rush] Move the rushstack repo to pnpm 12.13.0 #6136): it installs with--prefer-frozen-lockfile, so the pnpmfile hooks still run. Thebuild-tests-subspaceinstall proves this. No change needed.pnpmSyncPrepareAsync): it reads lockfile paths, not dependency specs. No change needed.Subspace.getPackageJsonInjectedDependenciesHash: it hashed the rawcatalog:specs. Fixed as described above.Tests:
SubspaceGlobalPnpmfileShim.test.tscovers:catalog:default;dependencies,optionalDependenciesandpeerDependenciesfields;SubspacePnpmfileConfiguration.test.tscoverscatalogsBySubspaceNameandsubspaceName: effective catalogs per subspace, the fallback tocommon/config/rush/pnpm-config.json, and that unrelated subspaces are omitted.Subspace.test.tscovers that the injected-dependencies hash follows catalog entry changes.logic/pnpm/test/repoWithInjectedSubspaceCatalogs/.Change file:
@microsoft/rush,patch(a bug fix in catalog support). The API report is unchanged; the new fields are on internal types.Commit 2:
[repo] Catalog prod/optional deps of injected default-subspace projectscatalog:in 23 projects, covering 70 packages.globalCatalogs.defaultincommon/config/rush/pnpm-config.json, kept sorted. The other 28 packages already had entries from [rushstack] Use pnpm catalogs for external dependency versions #6146. These match the specs and entries that [rushstack] Use pnpm catalogs for external dependency versions #6146 left out.globalCatalogscomment inpnpm-config.jsonnow says injected projects may usecatalog:because the shim resolves it.allowedAlternativeVersionsspecs,npm:aliases, dist-tags;@types/vscode(vsce rejectscatalog:),@pnpm/logger,@pnpm/lockfile.types;package.jsonfiles.rush checkreports no catalog/literal mismatches.tapable(1.1.3 / 2.2.1),source-map~0.6.1andtypescript5.9.3 inapi-extractor(all three areallowedAlternativeVersions),@pnpm/logger, and thenpm:aliases inrush-pnpm-kit-v8/9/10andrush-sdk.nonechange files, with empty comments.Lockfile churn (compared with #6146's head
266d15f735):defaultpnpm-lock.yaml: 0 packages or snapshots added, removed or changed. 0 importer version changes. 98 importer specifiers changed tocatalog:, and the 42 entries were added tocatalogs.default.build-tests-subspacepnpm-lock.yaml: byte-identical.default, 94 inbuild-tests-subspace) are the same as at [rushstack] Use pnpm catalogs for external dependency versions #6146's head; no sha512 hash was downgraded.pnpmfileChecksumis unchanged.repo-state.json:default:pnpmShrinkwrapHashandpnpmCatalogsHashchange.build-tests-subspace:pnpmCatalogsHashchanges because it falls back to the commonpnpm-config.json.packageJsonInjectedDependenciesHashchanges (013bef6→dcb9a4c) because of commit 1's catalog-aware hashing. Commit 2's resolved hash inputs are the same as commit 1's alone.Prerequisites before CI can pass
CI runs Rush through
install-run-rushwith the pinnedrushVersion5.181.0. That version has neither this fix nor the lower layers. Commit 2 cannot pass CI until all three of these happen:rushVersionbump to that release.The same blockers apply to #6146.
Notes and deviations
catalog-providerandcatalog-consumer, to avoidjest-haste-mapname collisions with other fixtures.build-tests-subspace'spackageJsonInjectedDependenciesHashchanges because of commit 1, not commit 2, as explained above.CI=true rush installfor thedefaultsubspace ran with the Rush built from [rushstack] Use pnpm catalogs for external dependency versions #6146's head. Under a dev Rush built from this branch, that install purgescommon/temp/default/node_modules, from which the dev Rush loads its own dependencies (a self-hosting artifact, not a bug).build-tests-subspaceran with the dev Rush from this branch.How it was tested
Unless noted otherwise, every command was run on the final head with a Rush built from this branch (
rush build --to @microsoft/rush, thennode apps/rush/lib-commonjs/start-dev.js …), using pnpm 12.13.0 and Node 22.23.2 on macOS.common/scriptswas refreshed temporarily for the dev Rush and restored before committing.heft test --production --cleaninlibraries/rush-libTarExecutable, which is known to fail locally onlyrush update, stripms-feedtarball URLs,rush updateagainCI=true rush install(both subspaces'last-install.flagdeleted)repo-state.json("Lockfile is up to date").defaultran with #6146's Rush; see the deviations aboverush check --subspace default,rush check --subspace build-tests-subspacerush build --production(full repo)rush change --verify -b origin/bmiddha-migrate-rushstack-to-pnpm-catalogs, andrush change --verifyrush-pnpm packfornode-core-library,heft,rush,lockfile-explorer, plusrush-libandapi-extractorcatalog:in any packedpackage.json. Every dependency spec is identical to the manifest at #6146's head. Specs newly cataloged and resolved back on pack:node-core-library8,heft5,rush-lib24,api-extractor6rush add -p strict-uri-encodeinlibraries/node-core-library, thenrush update(smoke test, reverted)"strict-uri-encode": "catalog:". Thebuild-tests-subspaceinstall succeeded, and the injectednode-core-librarysnapshot resolvedstrict-uri-encode: 2.0.0from catalog entry~2.0.0rush updateERR_PNPM_SPEC_NOT_SUPPORTED_BY_ANY_RESOLVER "@types/tapable@catalog:"inbuild-tests-subspace. After restoring commit 1,rush updatesucceeded with lockfiles andrepo-state.jsonunchangedA full build also rewrote some committed
build-testsAPI Extractor outputs: CRLF line endings, and atsdoc-metadataversion change from 7.58.11 to 7.59.4. The same happens at #6146's head, and those files are not part of this PR.Impacted documentation
None beyond the stack's catalog documentation: injected cross-subspace projects can now use
catalog:independenciesandoptionalDependencies.