Installs with the Rust pnpm engine always re-resolve: state-dependent generated manifests and missing dedupePeers option

[zkochan]

Findings from profiling bd install in the bit2 workspace (331 components, ~8.5k packages) with the Rust pnpm engine (@pnpm/napi). Every install was taking ~4 minutes even with an up-to-date lockfile, and rm -rf node_modules && bd install appeared to hang at Progress: resolved 0, reused N. The engine-side causes are being fixed in pnpm (see references at the bottom); this issue tracks the two findings that need fixes in Bit, because they force the engine into a full re-resolution on every install regardless of how fast the engine is.

1. Generated component manifests differ depending on whether node_modules exists

Bit's dependency detection produces different manifests for the same workspace state depending on the presence of node_modules:

• with node_modules present, the generated manifests for ui/api-diff-view, ui/hooks/use-bulk-paged-query, and ui/inline-config-compare include @apollo/client@^3.12.0;
• with node_modules absent, the same components' manifests do not include it — even though all three genuinely import/require @apollo/client in their source.

The engine's lockfile freshness check therefore flaps forever. Captured from the engine's staleness log (TRACE=pacquet::install=info):

• install with node_modules present, lockfile from a no-node_modules install: specifiers in the lockfile don't match specifiers in package.json: * 1 dependency was added: @apollo/client@^3.12.0
• install after rm -rf node_modules, lockfile from a with-node_modules install: ... * 1 dependency was removed: @apollo/client@^3.12.0

Because no lockfile can satisfy both states, every install after removing node_modules (and the next one after it is restored) runs a full re-resolution (~3–4 minutes through the engine today) instead of the frozen path. With the flap fixed, the measured frozen path is ~1s of engine time on a warm workspace and ~5.5s of engine materialization after rm -rf node_modules — the "hang" at resolved 0, reused N was the full re-resolution, not the store/link pipeline.

Expected fix: dependency detection / version attribution must be independent of installed state — a component that imports @apollo/client should get the same manifest entry whether or not node_modules exists.

2. lynx.js does not pass dedupePeers: true to the engine

Bit's existing lockfiles record settings.dedupePeers: true (generated by the previous engine integration). The current installOptions built in scopes/dependencies/pnpm/lynx.ts does not pass dedupePeers, so the engine resolves it to its default (false) and its freshness gate reports:

`dedupePeers` in the lockfile (true) doesn't match the current config (false)

→ every install of an existing Bit repo is treated as outdated, full-resolves, and rewrites the entire lockfile's peer suffixes (a ~50k-line diff), which then keeps churning.

Until now this was not fixable from Bit's side because the @pnpm/napi install options did not expose dedupePeers at all; that binding gap is fixed in pnpm/pnpm#13492. Once the @pnpm/napi release containing it ships, lynx.ts should pass dedupePeers: true alongside dedupePeerDependents: true.

Engine-side context (fixed or in progress on the pnpm side)

For completeness — these were found in the same investigation and are not Bit bugs, but they explain the remaining numbers:

• multi-importer peer-hoist discovery made from-scratch resolution quadratic — fixed and merged (perf(resolver): make multi-importer peer-hoist discovery linear pnpm/pnpm#13491);
• the per-manifest readPackage hook dispatch cost ~1 event-loop tick per manifest (~200s per full resolve at Bit's scale) — now batched; a quadratic per-package tree scan in the resolver — now indexed; overrides were recorded in random order every install — now order-preserving; all in fix(napi): deterministic overrides, hook-free install options, and large-workspace install fixes pnpm/pnpm#13492;
• remaining engine work (tracked on the pnpm side): resolution executes single-threaded, and lockfile-satisfied picks can still wait on per-package metadata refreshes. A full re-resolution at bit2's scale currently costs ~114s hook-free (~177s with the hook); the goal is double-digit seconds.

With both Bit-side fixes in place, bd install only pays resolution when dependencies actually change; warm repeat installs take ~1s of engine time and rm -rf node_modules && bd install ~5.5s.

---

Written by an agent (Claude Code, claude-fable-5), investigating together with Zoltan Kochan.

[zkochan]

From-scratch install benchmark findings (rm -rf node_modules pnpm-lock.yaml)

Follow-up findings from comparing a from-scratch stable bit install (TS pnpm v10 engine) against bd install (Rust @pnpm/napi engine) on the same 345-importer workspace:

1. networkConcurrency: 16 is the dominant from-scratch bottleneck

lynx.ts passes networkConcurrency: 16 to the engine. A from-scratch resolution makes ~3,400 metadata round trips against real registries (~150–200 ms each), so 16 concurrent requests cap throughput at ~100 req/s — whole-process stack sampling showed the engine idle (zero busy threads) for two-thirds of the resolution wall time, waiting on that semaphore.

Raising it to 64 (no other changes):

configuration	from-scratch resolution	full bd install
networkConcurrency: 16	38–62 s	131 s
networkConcurrency: 64	11.8–28 s	94.5 s
stable bit (TS pnpm v10)	—	140.8 s

No 429s and no retry-after-backoff events at 64 against either registry (npmjs + node-registry.bit.cloud). The engine-side share drops to ~33 s vs the TS engine's ~77 s (2.3×), and to ~17 s (4.5×) under policy-equivalent settings (see item 3).

2. Committed engineStrict + nodeVersion make from-scratch installs fail

workspace.jsonc pins "nodeVersion": "22.22.0" with "engineStrict": true. A from-scratch resolve picks jsdom@30.0.0 (transitive), whose engines require ^22.22.2 || ^24.15.0 || >=26.0.0, so the install hard-fails with ERR_PNPM_UNSUPPORTED_ENGINE — in either engine (the TS packageIsInstallable throws identically under engineStrict). Installs only succeed today where a lockfile pins jsdom lower, or in checkouts where those two lines were removed locally. Either hold jsdom back, raise the nodeVersion pin to ≥22.22.2, or drop engineStrict.

3. minimumReleaseAge is silently active under the Rust engine

lynx.ts passes minimumReleaseAge: options.minimumReleaseAge, which is undefined unless configured — and the pnpm v12 engine then applies its own default of 1440 minutes (24 h). TS pnpm v10 has no such policy, so Rust-engine installs pay maturity filtering, per-name full-metadata upgrade probes (node-registry.bit.cloud serves no per-version time field and answers every upgrade with 304), and extra revalidation — v10 installs pay none of it. This is also why fresh resolutions can hold back versions published in the last 24 h. Bit should decide the policy explicitly (pass 0 to match v10 behavior, or keep/raise the window deliberately).

---

Written by an agent (Claude Code, claude-fable-5).

[zkochan]

Follow-up: the exact source of networkConcurrency: 16, and where the fix belongs

Traced the value end to end:

• pnpm itself is not the source. Both engines default networkConcurrency to min(96, max(workers × 3, 64)) since June 2026 (perf: close the warm-resolve, symlink-churn, and download-concurrency gaps pnpm/pnpm#12329, raised in both the TypeScript and Rust stacks after the vlt.sh cold-cell benchmarks; an earlier April change, pacquet#302, had aligned a mistaken hardcoded 50 down to pnpm's then-formula while fixing socket-bounding bugs).
• Bit overrides that default for every install: Http.getNetworkConfig() (scopes/scope/network/http/http.ts) falls back to networkConcurrency: … ?? 16 when the user hasn't set network_concurrency in bit config. Because this layer always yields a number, the merged network config always carries 16, lynx forwards it, and the package manager's own default never applies — under the Rust engine and under stable's TS engine alike.

Suggested fix: drop the ?? 16 fallback (leave the key absent unless explicitly configured) so the package manager's default min(96, max(workers × 3, 64)) applies, or raise the fallback to match. Measured effect on a 345-importer workspace at 64: from-scratch resolution 38→12 s, full install 131→94.5 s, with zero HTTP 429/retry events against npmjs and node-registry.bit.cloud. This also speeds up stable (TS-engine) installs.

---

Written by an agent (Claude Code, claude-fable-5).