Skip to content

[rush] Resolve pnpm catalog references in injected cross-subspace dependencies - #6148

Draft
Bharat Middha (bmiddha) wants to merge 2 commits into
bmiddha-migrate-rushstack-to-pnpm-catalogsfrom
bmiddha-resolve-catalogs-in-injected-deps
Draft

Bharat Middha (bmiddha) wants to merge 2 commits into
bmiddha-migrate-rushstack-to-pnpm-catalogsfrom
bmiddha-resolve-catalogs-in-injected-deps

Conversation

@bmiddha

Copy link
Copy Markdown
Member

Part of stack #6134, above #6146.

Summary

Problem. A subspace can inject Rush projects from another subspace. The subspace pnpmfile shim rewrites those workspace: dependencies to file:<relative path>, so pnpm treats the injected projects as external packages and resolves their dependencies and optionalDependencies inside the consuming subspace's install. pnpm only resolves catalog: for the projects of its own workspace, so a catalog: spec in an injected project fails:

ERR_PNPM_SPEC_NOT_SUPPORTED_BY_ANY_RESOLVER  "@types/tapable@catalog:" isn't supported by any available resolver.

Even if pnpm could resolve it, it would use the consuming subspace's catalog, which is the wrong one.

In this repo, build-tests-subspace injects 35 default-subspace projects, transitively. #6146 worked around the error by keeping 98 prod/optional specs in that closure literal. That left a trap: after #6141, rush add of a prod dependency into any of those projects writes catalog:, and the build-tests-subspace install breaks.

Fix. This PR has two commits.

  1. [rush]: in readPackage, the subspace pnpmfile shim now replaces catalog: and catalog:<name> in injected cross-subspace Rush projects with the literal entry from the catalogs of the project's own subspace.
  2. [repo]: catalogs the 98 literal prod/optional specs in the injected closure, reversing the [rushstack] Use pnpm catalogs for external dependency versions #6146 workaround.

Details

Commit 1: [rush] Resolve pnpm catalogs in injected cross-subspace deps

  • Settings (IPnpmfile.ts, SubspacePnpmfileConfiguration.ts):
    • ISubspacePnpmfileShimSettings gains catalogsBySubspaceName?: Record<string, PnpmCatalogs>, and IWorkspaceProjectInfo gains subspaceName?. Both are optional and additive; the settings and the shim are always generated by the same Rush version.
    • The catalogs come from getEffectivePnpmCatalogs for each other subspace that has an injected project: the subspace's pnpm-config.json, falling back to common/config/rush/pnpm-config.json ([rush] Use the common pnpm-config.json fallback for every pnpm catalog consumer #6138).
  • Shim (SubspaceGlobalPnpmfileShim.ts): readPackage calls resolveOtherSubspaceCatalogReferences(pkg) before the existing workspace: → file: rewrite and before the user pnpmfile.
    • Which packages: only packages listed in settings.workspaceProjects, which holds only Rush projects from other subspaces. The current subspace's own importers are never rewritten: pnpm resolves those natively, and their lockfile importer specifiers keep catalog:.
    • Which fields: dependencies, optionalDependencies and peerDependencies, the fields pnpm resolves for an installed package. devDependencies of an external package are not installed.
    • Which specs: catalog: and catalog:<name> (catalog:default is the same as catalog:), parsed with the helpers from [rush] Resolve PNPM catalog references in implicitly preferred versions #6139 (PnpmCatalogReference.ts).
    • Errors: a missing catalog map for the subspace, a missing named catalog, or a missing entry each throws an error naming the project, its subspace, the dependency and spec, and the catalog.
    • A reset() export clears the shim's module state, for tests.
  • Injected-dependencies hash (api/Subspace.ts): getPackageJsonInjectedDependenciesHash now resolves catalog refs of projects outside the subspace before hashing. Without this, editing a catalog entry used by an injected project would not invalidate the consuming subspace's install.
  • Lockfile: the rewritten specs are inputs to resolution only. The file: packages and snapshots resolve to the same versions as the literals did, so the lockfile content is unchanged. This repo's build-tests-subspace lockfile is byte-identical.

Other Rush code that reads injected manifests. I checked each candidate for mis-handling of catalog::

  • PnpmShrinkwrapFile.isWorkspaceProjectModifiedAsync: it compares only the subspace's own importers, which pnpm resolves natively and which still contain catalog:. No change needed.
  • BaseInstallManager, pnpm >= 12 injected path ([rush] Move the rushstack repo to pnpm 12.13.0 #6136): it installs with --prefer-frozen-lockfile, so the pnpmfile hooks still run. The build-tests-subspace install proves this. No change needed.
  • pnpm-sync (pnpmSyncPrepareAsync): it reads lockfile paths, not dependency specs. No change needed.
  • Subspace.getPackageJsonInjectedDependenciesHash: it hashed the raw catalog: specs. Fixed as described above.

Tests:

  • SubspaceGlobalPnpmfileShim.test.ts covers:
    • default and named catalogs, including catalog:default;
    • the dependencies, optionalDependencies and peerDependencies fields;
    • the current subspace's own projects and non-Rush packages, which are left untouched;
    • the three error cases.
  • SubspacePnpmfileConfiguration.test.ts covers catalogsBySubspaceName and subspaceName: effective catalogs per subspace, the fallback to common/config/rush/pnpm-config.json, and that unrelated subspaces are omitted.
  • Subspace.test.ts covers that the injected-dependencies hash follows catalog entry changes.
  • New fixture: logic/pnpm/test/repoWithInjectedSubspaceCatalogs/.

Change file: @microsoft/rush, patch (a bug fix in catalog support). The API report is unchanged; the new fields are on internal types.

Commit 2: [repo] Catalog prod/optional deps of injected default-subspace projects

  • 98 prod/optional specs → catalog: in 23 projects, covering 70 packages.
  • 42 entries re-added to globalCatalogs.default in common/config/rush/pnpm-config.json, kept sorted. The other 28 packages already had entries from [rushstack] Use pnpm catalogs for external dependency versions #6146. These match the specs and entries that [rushstack] Use pnpm catalogs for external dependency versions #6146 left out.
  • The globalCatalogs comment in pnpm-config.json now says injected projects may use catalog: because the shim resolves it.
  • Same exclusions as [rushstack] Use pnpm catalogs for external dependency versions #6146:
    • allowedAlternativeVersions specs, npm: aliases, dist-tags;
    • @types/vscode (vsce rejects catalog:), @pnpm/logger, @pnpm/lockfile.types;
    • peers, workspace/decoupled-local deps, autoinstallers, and non-project package.json files.
  • Consumers migrated together: all consumers of each package were migrated together, so rush check reports no catalog/literal mismatches.
  • 15 closure literals remain, all excluded above: tapable (1.1.3 / 2.2.1), source-map ~0.6.1 and typescript 5.9.3 in api-extractor (all three are allowedAlternativeVersions), @pnpm/logger, and the npm: aliases in rush-pnpm-kit-v8/9/10 and rush-sdk.
  • 20 none change files, with empty comments.

Lockfile churn (compared with #6146's head 266d15f735):

  • default pnpm-lock.yaml: 0 packages or snapshots added, removed or changed. 0 importer version changes. 98 importer specifiers changed to catalog:, and the 42 entries were added to catalogs.default.
  • build-tests-subspace pnpm-lock.yaml: byte-identical.
  • Integrity hashes: unchanged. The remaining sha1 counts (220 in default, 94 in build-tests-subspace) are the same as at [rushstack] Use pnpm catalogs for external dependency versions #6146's head; no sha512 hash was downgraded. pnpmfileChecksum is unchanged.
  • repo-state.json:
    • default: pnpmShrinkwrapHash and pnpmCatalogsHash change.
    • build-tests-subspace: pnpmCatalogsHash changes because it falls back to the common pnpm-config.json. packageJsonInjectedDependenciesHash changes (013bef6 → dcb9a4c) because of commit 1's catalog-aware hashing. Commit 2's resolved hash inputs are the same as commit 1's alone.

Prerequisites before CI can pass

CI runs Rush through install-run-rush with the pinned rushVersion 5.181.0. That version has neither this fix nor the lower layers. Commit 2 cannot pass CI until all three of these happen:

  1. A Rush release that includes commit 1 and the lower layers of #6134.
  2. A rushVersion bump to that release.
  3. The npm feed serving pnpm 12.13.0. It currently serves up to 12.10.1.

The same blockers apply to #6146.

Notes and deviations

  • The fixture's injected packages are named catalog-provider and catalog-consumer, to avoid jest-haste-map name collisions with other fixtures.
  • build-tests-subspace's packageJsonInjectedDependenciesHash changes because of commit 1, not commit 2, as explained above.
  • The forced CI=true rush install for the default subspace ran with the Rush built from [rushstack] Use pnpm catalogs for external dependency versions #6146's head. Under a dev Rush built from this branch, that install purges common/temp/default/node_modules, from which the dev Rush loads its own dependencies (a self-hosting artifact, not a bug). build-tests-subspace ran with the dev Rush from this branch.

How it was tested

Unless noted otherwise, every command was run on the final head with a Rush built from this branch (rush build --to @microsoft/rush, then node apps/rush/lib-commonjs/start-dev.js …), using pnpm 12.13.0 and Node 22.23.2 on macOS. common/scripts was refreshed temporarily for the dev Rush and restored before committing.

Command Result
heft test --production --clean in libraries/rush-lib ✅ 2022 passed. The only failure is TarExecutable, which is known to fail locally only
rush update, strip ms-feed tarball URLs, rush update again ✅ Stable. Zero resolved-version churn and sha512 hashes kept (see the churn summary above)
CI=true rush install (both subspaces' last-install.flag deleted) ✅ No changes to lockfiles or repo-state.json ("Lockfile is up to date"). default ran with #6146's Rush; see the deviations above
rush check --subspace default, rush check --subspace build-tests-subspace ✅ "Found no mis-matching dependencies!" (both)
rush build --production (full repo) ✅ Exit 0: 183 succeeded, 17 no-op, 0 warnings, 0 failures
rush change --verify -b origin/bmiddha-migrate-rushstack-to-pnpm-catalogs, and rush change --verify ✅ Both passed
rush-pnpm pack for node-core-library, heft, rush, lockfile-explorer, plus rush-lib and api-extractor ✅ No catalog: in any packed package.json. Every dependency spec is identical to the manifest at #6146's head. Specs newly cataloged and resolved back on pack: node-core-library 8, heft 5, rush-lib 24, api-extractor 6
rush add -p strict-uri-encode in libraries/node-core-library, then rush update (smoke test, reverted) ✅ Wrote "strict-uri-encode": "catalog:". The build-tests-subspace install succeeded, and the injected node-core-library snapshot resolved strict-uri-encode: 2.0.0 from catalog entry ~2.0.0
Commit 1 reverted temporarily (rush-lib rebuilt), then rush update ✅ Failed as expected: ERR_PNPM_SPEC_NOT_SUPPORTED_BY_ANY_RESOLVER "@types/tapable@catalog:" in build-tests-subspace. After restoring commit 1, rush update succeeded with lockfiles and repo-state.json unchanged

A full build also rewrote some committed build-tests API Extractor outputs: CRLF line endings, and a tsdoc-metadata version change from 7.58.11 to 7.59.4. The same happens at #6146's head, and those files are not part of this PR.

Impacted documentation

None beyond the stack's catalog documentation: injected cross-subspace projects can now use catalog: in dependencies and optionalDependencies.

PNPM resolves "catalog:" only for its own workspace importers. A Rush
project from another subspace is injected as an external "file:"
package, so PNPM rejects its catalog references (or would use the
wrong subspace's catalog).

The subspace pnpmfile shim now rewrites catalog references in
dependencies, optionalDependencies and peerDependencies of projects
from other subspaces, using the effective catalogs of the subspace
that contains each project. The catalogs are passed through new
optional pnpmfile settings. The injected dependencies hash also hashes
the resolved versions, so catalog edits trigger a reinstall.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Now that the subspace pnpmfile shim resolves `catalog:` references in
injected cross-subspace Rush projects, migrate the 98 literal
dependencies/optionalDependencies specs that #6146 kept in the
build-tests-subspace injected closure (23 projects) to `catalog:`, and
add the 42 corresponding entries to globalCatalogs.default.

Resolved versions are unchanged: lockfile packages/snapshots are
identical, and sha512 integrity hashes are preserved.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-project-automation github-project-automation Bot moved this to Needs triage in Bug Triage Oct 15, 2026
@bmiddha
Bharat Middha (bmiddha) added this pull request to stack #6134 October 15, 2026 06:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs triage

Development

Successfully merging this pull request may close these issues.

1 participant