From d2efd9ce6efb9512dd235e1a01a5dc858bcfc5fd Mon Sep 17 00:00:00 2001 From: Aurora Scharff <66901228+aurorascharff@users.noreply.github.com> Date: Fri, 21 Aug 2026 16:01:28 +0200 Subject: [PATCH 1/7] Improve Partial Prefetching adoption checks (#97637) ## Summary Dogfooding the Partial Prefetching adoption skill on v0 exposed two preservation checks that should be part of the main workflow: - trace custom Link wrapper consumers and distinguish declarative prefetch props from imperative hover or touch prefetching - treat existing instant() assertions as preservation contracts, and expand caching only to the smallest coherent rendered subtree when a data-only cache does not restore the prefetched UI ## Verification - pnpm prettier --with-node-modules --ignore-path .prettierignore --check skills/next-partial-prefetching-adoption/SKILL.md - Commit hook lint-staged formatting passed - Not run: quick_validate.py because PyYAML is unavailable in the local skill validator runtime --- skills/next-partial-prefetching-adoption/SKILL.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/skills/next-partial-prefetching-adoption/SKILL.md b/skills/next-partial-prefetching-adoption/SKILL.md index 82696915d50b..f09895b653d9 100644 --- a/skills/next-partial-prefetching-adoption/SKILL.md +++ b/skills/next-partial-prefetching-adoption/SKILL.md @@ -56,7 +56,7 @@ If `partialPrefetching: true` is already set in `next.config.ts`, the app is ado The work is identical either way — only the commit boundaries differ. Default by app size: one branch for a handful of links, route by route when the audit is big enough that reviewers need smaller diffs. Note the choice in your report. -Enumerate the prefetch sites across the whole source tree, not only `app/` — they often live in `src/components` or shared UI packages: `rg -n '\bprefetch\b|router\.prefetch' -g '*.tsx' -g '*.jsx' .`. Keep the `` and bare-prop matches (a bare prop is `true`) as the over-prefetching links this audit adopts destinations for, and drop `prefetch={false}` and other values. Also audit existing imperative [`router.prefetch()`](https://nextjs.org/docs/app/api-reference/functions/use-router#userouter) call sites with the same table, because they can be preserving the same "fetch before navigation" behavior and have no dev insight. For new navigation prefetching, prefer [``](https://nextjs.org/docs/app/api-reference/components/link), which the docs call the primary navigation API; use [`router.prefetch()`](https://nextjs.org/docs/app/guides/prefetching#manual-prefetch) only for manual prefetching. If the app already passes an internal `kind` option, treat that as existing implementation detail, not a pattern to spread. If nothing matches, check for a custom link wrapper before calling the audit empty. If there's still nothing, say so in your report and move on to [step 2](#step-2-enable-the-flag). +Enumerate the prefetch sites across the whole source tree, not only `app/` — they often live in `src/components` or shared UI packages: `rg -n '\bprefetch\b|router\.prefetch' -g '*.tsx' -g '*.jsx' .`. Keep the `` and bare-prop matches (a bare prop is `true`) as the over-prefetching links this audit adopts destinations for, and drop `prefetch={false}` and other values. Also audit existing imperative [`router.prefetch()`](https://nextjs.org/docs/app/api-reference/functions/use-router#userouter) call sites with the same table, because they can be preserving the same "fetch before navigation" behavior and have no dev insight. For new navigation prefetching, prefer [``](https://nextjs.org/docs/app/api-reference/components/link), which the docs call the primary navigation API; use [`router.prefetch()`](https://nextjs.org/docs/app/guides/prefetching#manual-prefetch) only for manual prefetching. If the app already passes an internal `kind` option, treat that as existing implementation detail, not a pattern to spread. Always inspect custom Link wrappers and trace their consumers: a wrapper can call `router.prefetch()` on hover or touch while a consumer omits `prefetch` or passes `prefetch={false}`. Record the declarative and imperative behavior separately, and use keyboard activation when verifying the declarative path so hover prefetching does not mask it. If nothing matches, say so in your report and move on to [step 2](#step-2-enable-the-flag). Then, for each one: @@ -80,6 +80,8 @@ Then, for each one: 3. **Preserve what that prefetch delivered.** The guide's [audit table](https://nextjs.org/docs/app/guides/adopting-partial-prefetching#auditing-link-prefetchtrue-calls) is the canonical decision — fetch it and apply the matching row. Caching uncached content is the judgment call in that table: trace where the data comes from and what freshness and revalidation it needs, per the [`use cache`](https://nextjs.org/docs/app/api-reference/directives/use-cache) docs, and ask the user when the answer isn't clear-cut. The URL-data routes you marked in the previous item wait for step 5. + If the repository already has `instant()` e2e coverage for a destination, run it before editing and preserve its assertion as the contract. A successful build or completed navigation does not prove that the same UI was prefetched. If caching the primary data loader still leaves only a fallback inside `instant()`, inspect rendered descendants and providers for dynamic work, then expand the cache only to the smallest coherent rendered subtree that restores the contract. + > **If you add `use cache`, verify under `next start`, not only the build.** A `cookies()`/`headers()`/session read anywhere in the cached call tree throws at request time while `next build` passes clean. See [`use cache`](https://nextjs.org/docs/app/api-reference/directives/use-cache). ## step 2: enable the flag From ba9f073cc28919ba894d64efaaa82ee50824c0fb Mon Sep 17 00:00:00 2001 From: David Alexandru Ilie Date: Fri, 21 Aug 2026 16:07:42 +0200 Subject: [PATCH 2/7] Add a Turbopack error for missing root layouts (#97639) ## Summary Turbopack currently builds an App Router page without a root layout, leaving the problem to surface later at runtime. Report it while building the app structure instead: `route/page.js doesn't have a root layout. To fix this error, make sure every page has a root layout.` The check uses the layouts already inherited during directory-tree traversal, so route groups and apps with multiple root layouts keep working. Route handlers are unaffected. Webpack behavior is intentionally unchanged in this PR. ## Verification - `HEADLESS=true pnpm test-dev-turbo test/e2e/app-dir/create-root-layout/create-root-layout.test.ts` - `HEADLESS=true pnpm test-start-turbo test/e2e/app-dir/create-root-layout/create-root-layout.test.ts` - `HEADLESS=true pnpm test-dev-webpack test/e2e/app-dir/create-root-layout/create-root-layout.test.ts` - `HEADLESS=true pnpm test-start-webpack test/e2e/app-dir/create-root-layout/create-root-layout.test.ts` - `HEADLESS=true pnpm test-dev-turbo test/e2e/app-dir/root-layout/root-layout.test.ts` --- crates/next-core/src/app_structure.rs | 48 ++++++++++++++++++- .../app-edge-invalid-reexport.test.ts | 1 + .../create-root-layout.test.ts | 39 ++++++++++++++- test/e2e/config-turbopack/index.test.ts | 5 ++ 4 files changed, 91 insertions(+), 2 deletions(-) diff --git a/crates/next-core/src/app_structure.rs b/crates/next-core/src/app_structure.rs index 84e9f6723d2f..160ed9ef1354 100644 --- a/crates/next-core/src/app_structure.rs +++ b/crates/next-core/src/app_structure.rs @@ -898,6 +898,43 @@ impl Issue for DuplicateParallelRouteIssue { } } +#[turbo_tasks::value] +struct MissingRootLayoutIssue { + app_dir: FileSystemPath, + page_path: FileSystemPath, +} + +#[async_trait] +#[turbo_tasks::value_impl] +impl Issue for MissingRootLayoutIssue { + async fn file_path(&self) -> Result { + Ok(self.page_path.clone()) + } + + fn stage(&self) -> IssueStage { + IssueStage::AppStructure + } + + fn severity(&self) -> IssueSeverity { + IssueSeverity::Error + } + + async fn title(&self) -> Result { + let page_path = self + .app_dir + .get_path_to(&self.page_path) + .context("page should be within the app directory")?; + + Ok(StyledString::Text( + format!( + "{page_path} doesn't have a root layout. To fix this error, make sure every page \ + has a root layout." + ) + .into(), + )) + } +} + #[turbo_tasks::value] struct MissingDefaultParallelRouteIssue { app_dir: FileSystemPath, @@ -1591,7 +1628,16 @@ async fn directory_tree_to_entrypoints_internal_untraced( root_params }; - if modules.page.is_some() { + if let Some(page_path) = &modules.page { + if root_layouts.await?.is_empty() { + MissingRootLayoutIssue { + app_dir: app_dir.clone(), + page_path: page_path.clone(), + } + .resolved_cell() + .emit(); + } + let app_path = AppPath::from(app_page.clone()); let loader_tree = *directory_tree_to_loader_tree( diff --git a/test/e2e/app-dir/app-edge/app-edge-invalid-reexport.test.ts b/test/e2e/app-dir/app-edge/app-edge-invalid-reexport.test.ts index 2ff08c749685..85cece609e7c 100644 --- a/test/e2e/app-dir/app-edge/app-edge-invalid-reexport.test.ts +++ b/test/e2e/app-dir/app-edge/app-edge-invalid-reexport.test.ts @@ -4,6 +4,7 @@ import path from 'path' describe('app-dir edge SSR invalid reexport', () => { const { next, isNextDev, skipped } = nextTestSetup({ files: { + 'app/layout.tsx': new FileRef(path.join(__dirname, 'app', 'layout.tsx')), 'app/export': new FileRef(path.join(__dirname, 'app', 'export')), 'app/export/inherit/page.tsx': "export { default, runtime, preferredRegion } from '../basic/page'", diff --git a/test/e2e/app-dir/create-root-layout/create-root-layout.test.ts b/test/e2e/app-dir/create-root-layout/create-root-layout.test.ts index 430e116dcd50..8853aa0f5041 100644 --- a/test/e2e/app-dir/create-root-layout/create-root-layout.test.ts +++ b/test/e2e/app-dir/create-root-layout/create-root-layout.test.ts @@ -1,6 +1,6 @@ import path from 'path' import { FileRef, nextTestSetup } from 'e2e-utils' -import { check } from 'next-test-utils' +import { check, retry } from 'next-test-utils' import stripAnsi from 'strip-ansi' // Skip on Turbopack because the user should create the layout manually @@ -224,3 +224,40 @@ import stripAnsi from 'strip-ansi' } } ) +;(process.env.IS_TURBOPACK_TEST ? describe : describe.skip)( + 'app-dir missing root layout', + () => { + const { next, isNextDev, skipped } = nextTestSetup({ + files: { + app: new FileRef(path.join(__dirname, 'app')), + 'next.config.js': new FileRef(path.join(__dirname, 'next.config.js')), + }, + skipDeployment: true, + skipStart: true, + }) + + if (skipped) return + + it('reports a compiler error without modifying the app', async () => { + if (isNextDev) { + await next.start() + + const response = await next.fetch('/route') + expect(response.status).toBe(500) + + await retry(async () => { + expect(stripAnsi(next.cliOutput)).toContain( + "route/page.js doesn't have a root layout. To fix this error, make sure every page has a root layout." + ) + }) + } else { + await expect(next.start()).rejects.toThrow('next build failed') + expect(stripAnsi(next.cliOutput)).toContain( + "route/page.js doesn't have a root layout. To fix this error, make sure every page has a root layout." + ) + } + + expect(await next.hasFile('app/layout.js')).toBe(false) + }) + } +) diff --git a/test/e2e/config-turbopack/index.test.ts b/test/e2e/config-turbopack/index.test.ts index 9e94b29a9b68..af35036175c9 100644 --- a/test/e2e/config-turbopack/index.test.ts +++ b/test/e2e/config-turbopack/index.test.ts @@ -6,6 +6,11 @@ const WARNING_MESSAGE = `ERROR: This build is using Turbopack, with a \`webpack\ const itif = (condition: boolean) => (condition ? it : it.skip) const page = { + 'app/layout.js': ` +export default function RootLayout({ children }) { + return {children} +} +`, 'app/page.js': ` export default function Page() { return

hello world

From 8482890aca9be98b642fc85957ccc177d821eb1a Mon Sep 17 00:00:00 2001 From: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com> Date: Fri, 21 Aug 2026 17:37:46 +0200 Subject: [PATCH 3/7] test: make app-document-import-order more robust (#97682) - The `brower.eval(callback)` version was a bit weird, use beforePageLoad as we do everywhere else - Properly parse the URL and match against the pathname, to not break with `?dpl` --- .../app-document-import-order.test.ts | 43 +++++++++++-------- 1 file changed, 24 insertions(+), 19 deletions(-) diff --git a/test/e2e/app-document-import-order/app-document-import-order.test.ts b/test/e2e/app-document-import-order/app-document-import-order.test.ts index b248c886a9a1..661630646aac 100644 --- a/test/e2e/app-document-import-order/app-document-import-order.test.ts +++ b/test/e2e/app-document-import-order/app-document-import-order.test.ts @@ -16,32 +16,37 @@ describe('Root components import order', () => { expect($(sideEffectCall).text()).toEqual(expectSideEffectsOrder[index]) }) }) + // Only asserted for production builds: in development each entry is chunked from its own // per-page module graph, which can still merge a shared module into per-entry units. ;(isNextDev ? it.skip : it)( 'loads modules shared by _app and the page only once', async () => { - const browser = await next.browser('/', { waitHydration: false }) - const markerCount = await browser.eval(async () => { - const chunkUrls = [ - ...new Set( - performance - .getEntriesByType('resource') - .map((entry) => entry.name) - .filter( - (url) => url.includes('/_next/static/') && url.endsWith('.js') - ) - ), - ] - const chunks = await Promise.all( - chunkUrls.map((url) => fetch(url).then((response) => response.text())) - ) - return chunks.filter((chunk) => - chunk.includes('APP_PAGE_SHARED_MODULE_MARKER') - ).length + const requests: Set = new Set() + await next.browser('/', { + beforePageLoad(page) { + page.on('request', (request) => { + const url = new URL(request.url(), next.url) + if ( + url.pathname.startsWith('/_next/static/') && + url.pathname.endsWith('.js') + ) { + requests.add(url.href) + } + }) + }, }) - expect(markerCount).toBe(1) + const chunks = await Promise.all( + [...requests].map((url) => + fetch(url).then((response) => response.text()) + ) + ) + const matchingChunks = chunks.filter((chunk) => + chunk.includes('APP_PAGE_SHARED_MODULE_MARKER') + ) + + expect(matchingChunks.length).toBe(1) } ) From 1c86b8a4fa3bb8795e25fd49bbbd10844dd83263 Mon Sep 17 00:00:00 2001 From: Luke Sandberg Date: Fri, 21 Aug 2026 08:40:43 -0700 Subject: [PATCH 4/7] turbo-tasks: add scope_unbounded, a scoped execution primitive that allows more work to be discovered (#95974) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What Adds `scope_unbounded `, a parallel scope to turbo-tasks: jobs run on a shared work-queue, any job may enqueue more, and the pool is drained by the runtime worker threads plus the calling thread until empty. ## Why The garbage collector (later in this stack) needs to process a pool of work that *discovers more work as it runs* — collecting a task cascades into collecting newly-parentless children. A fixed `parallel::for_each` can't express that. `scope_self_feeding` is the general primitive for "parallel pool where jobs spawn jobs". ## Design notes - **Deadlock-safe on thread-limited / contended runtimes.** Helper workers are a pure optimization: the calling thread always makes progress on the shared queue on its own, so the pool completes even when no worker threads are available. - Supports growing/shrinking the set of workers pulling from the queue based on the amount of work available. - Supports a simple mostly lock free way to aggregate data from the tasks - Supports a way for tasks to abort the whole queue, which is important for making GC interruptible ## Testing There are a fair number of new unit tests and i have done some runs under miri --- Cargo.lock | 1 + Cargo.toml | 1 + .../turbo-tasks-backend/src/backend/mod.rs | 4 +- .../src/backend/operation/connect_children.rs | 4 +- turbopack/crates/turbo-tasks/Cargo.toml | 1 + turbopack/crates/turbo-tasks/src/lib.rs | 4 +- turbopack/crates/turbo-tasks/src/parallel.rs | 18 +- .../src/{scope.rs => scope_bounded.rs} | 43 +- .../crates/turbo-tasks/src/scope_unbounded.rs | 977 ++++++++++++++++++ 9 files changed, 1023 insertions(+), 30 deletions(-) rename turbopack/crates/turbo-tasks/src/{scope.rs => scope_bounded.rs} (91%) create mode 100644 turbopack/crates/turbo-tasks/src/scope_unbounded.rs diff --git a/Cargo.lock b/Cargo.lock index ed64efe1791f..65af2c807eb6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -10083,6 +10083,7 @@ dependencies = [ "either", "erased-serde", "event-listener", + "fixedbitset", "futures", "indexmap 2.13.0", "parking_lot", diff --git a/Cargo.toml b/Cargo.toml index 0b765ff566b8..8f8a0a1519e7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -274,6 +274,7 @@ dhat = { version = "0.3.2" } dunce = "1.0.3" either = "1.15.0" erased-serde = "0.4.5" +fixedbitset = "0.5.7" flate2 = "1.0.28" fs-err = "3.1.1" futures = "0.3.31" diff --git a/turbopack/crates/turbo-tasks-backend/src/backend/mod.rs b/turbopack/crates/turbo-tasks-backend/src/backend/mod.rs index a24db97efaf6..2925e74b6d86 100644 --- a/turbopack/crates/turbo-tasks-backend/src/backend/mod.rs +++ b/turbopack/crates/turbo-tasks-backend/src/backend/mod.rs @@ -44,7 +44,7 @@ use turbo_tasks::{ macro_helpers::NativeFunction, message_queue::{TimingEvent, TraceEvent}, registry::get_value_type, - scope::scope_and_block, + scope_bounded::scope_bounded, task_statistics::TaskStatisticsApi, trace::TraceRawVcs, util::{IdFactoryWithReuse, good_chunk_size, into_chunks}, @@ -2522,7 +2522,7 @@ impl TurboTasksBackend { if output_dependent_tasks.len() > DEPENDENT_TASKS_DIRTY_PARALLELIZATION_THRESHOLD { let chunk_size = good_chunk_size(output_dependent_tasks.len()); let chunks = into_chunks(output_dependent_tasks.to_vec(), chunk_size); - let _ = scope_and_block(chunks.len(), |scope| { + let _ = scope_bounded(chunks.len(), |scope| { for chunk in chunks { let child_ctx = ctx.child_context(); #[cfg(feature = "task_dirty_cause")] diff --git a/turbopack/crates/turbo-tasks-backend/src/backend/operation/connect_children.rs b/turbopack/crates/turbo-tasks-backend/src/backend/operation/connect_children.rs index d5335b32560f..b2d4b4f72ad7 100644 --- a/turbopack/crates/turbo-tasks-backend/src/backend/operation/connect_children.rs +++ b/turbopack/crates/turbo-tasks-backend/src/backend/operation/connect_children.rs @@ -2,7 +2,7 @@ use rustc_hash::FxHashSet; use smallvec::SmallVec; use turbo_tasks::{ TaskId, - scope::scope_and_block, + scope_bounded::scope_bounded, util::{good_chunk_size, into_chunks}, }; @@ -139,7 +139,7 @@ pub fn connect_children( if len >= CONNECT_CHILDREN_PARALLIZATION_THRESHOLD { let new_follower_ids = new_follower_ids.into_vec(); let chunk_size = good_chunk_size(len); - let _ = scope_and_block(len.div_ceil(chunk_size), |scope| { + let _ = scope_bounded(len.div_ceil(chunk_size), |scope| { for chunk in into_chunks(new_follower_ids, chunk_size) { let upper_ids = &upper_ids; let child_ctx = ctx.child_context(); diff --git a/turbopack/crates/turbo-tasks/Cargo.toml b/turbopack/crates/turbo-tasks/Cargo.toml index efc022d8f2bc..e51681a78a26 100644 --- a/turbopack/crates/turbo-tasks/Cargo.toml +++ b/turbopack/crates/turbo-tasks/Cargo.toml @@ -33,6 +33,7 @@ dashmap = { workspace = true } either = { workspace = true } erased-serde = { workspace = true } event-listener = "5.4.0" +fixedbitset = { workspace = true } futures = { workspace = true } indexmap = { workspace = true, features = ["serde"] } parking_lot = { workspace = true, features = ["serde"]} diff --git a/turbopack/crates/turbo-tasks/src/lib.rs b/turbopack/crates/turbo-tasks/src/lib.rs index 3b25a4a06d62..d1c2c8278058 100644 --- a/turbopack/crates/turbo-tasks/src/lib.rs +++ b/turbopack/crates/turbo-tasks/src/lib.rs @@ -6,6 +6,7 @@ #![feature(arbitrary_self_types)] #![feature(arbitrary_self_types_pointers)] #![feature(ptr_metadata)] +#![feature(exclusive_wrapper)] #![feature(sync_unsafe_cell)] #![feature(async_fn_traits)] #![feature(impl_trait_in_assoc_type)] @@ -47,7 +48,8 @@ mod priority_runner; mod read_options; mod read_ref; pub mod registry; -pub mod scope; +pub mod scope_bounded; +pub mod scope_unbounded; mod serialization_invalidation; pub mod small_duration; mod spawn; diff --git a/turbopack/crates/turbo-tasks/src/parallel.rs b/turbopack/crates/turbo-tasks/src/parallel.rs index 047612bdf456..5e938843f499 100644 --- a/turbopack/crates/turbo-tasks/src/parallel.rs +++ b/turbopack/crates/turbo-tasks/src/parallel.rs @@ -13,7 +13,7 @@ use std::{ }; use crate::{ - scope::scope_and_block, + scope_bounded::scope_bounded, util::{Chunk, good_chunk_size, into_chunks}, }; @@ -76,7 +76,7 @@ where return; }; let f = &f; - let _results = scope_and_block(chunk_count, |scope| { + let _results = scope_bounded(chunk_count, |scope| { for chunk in items.chunks(chunk_size) { scope.spawn(move || { for item in chunk { @@ -102,7 +102,7 @@ where return; }; let f = &f; - let _results = scope_and_block(chunk_count, |scope| { + let _results = scope_bounded(chunk_count, |scope| { for chunk in into_chunks(items, chunk_size) { scope.spawn(move || { // SAFETY: Even when f() panics we drop all items in the chunk. @@ -133,7 +133,7 @@ where return Ok(()); }; let f = &f; - scope_and_block(chunk_count, |scope| { + scope_bounded(chunk_count, |scope| { for chunk in items.chunks(chunk_size) { scope.spawn(move || { for item in chunk { @@ -165,7 +165,7 @@ where return Ok(()); }; let f = &f; - scope_and_block(chunk_count, |scope| { + scope_bounded(chunk_count, |scope| { for chunk in items.chunks_mut(chunk_size) { scope.spawn(move || { for item in chunk { @@ -197,7 +197,7 @@ where return Ok(()); }; let f = &f; - scope_and_block(chunk_count, |scope| { + scope_bounded(chunk_count, |scope| { for chunk in into_chunks(items, chunk_size) { scope.spawn(move || { for item in chunk { @@ -227,7 +227,7 @@ where return Result::from_iter(items.iter().map(f)); }; let f = &f; - scope_and_block(chunk_count, |scope| { + scope_bounded(chunk_count, |scope| { for chunk in items.chunks(chunk_size) { scope.spawn(move || chunk.iter().map(f).collect::>()) } @@ -253,7 +253,7 @@ where return Result::from_iter(items.into_iter().map(f)); }; let f = &f; - scope_and_block(chunk_count, |scope| { + scope_bounded(chunk_count, |scope| { for chunk in into_chunks(items, chunk_size) { scope.spawn(move || chunk.map(f).collect::>()) } @@ -280,7 +280,7 @@ where return Result::from_iter(into_chunks(items, len).map(f)); }; let f = &f; - scope_and_block(chunk_count, |scope| { + scope_bounded(chunk_count, |scope| { for chunk in into_chunks(items, chunk_size) { scope.spawn(move || f(chunk)) } diff --git a/turbopack/crates/turbo-tasks/src/scope.rs b/turbopack/crates/turbo-tasks/src/scope_bounded.rs similarity index 91% rename from turbopack/crates/turbo-tasks/src/scope.rs rename to turbopack/crates/turbo-tasks/src/scope_bounded.rs index 249c37bc69a5..4f620ebdbe8d 100644 --- a/turbopack/crates/turbo-tasks/src/scope.rs +++ b/turbopack/crates/turbo-tasks/src/scope_bounded.rs @@ -1,4 +1,12 @@ -//! A scoped tokio spawn implementation that allow a non-'static lifetime for tasks. +//! Bounded scoped parallelism: the number of tasks is known before the scope starts. +//! +//! [`scope_bounded`] takes that count up front, hands the caller a [`Scope`] to spawn each task +//! onto, and returns their results as an iterator. Tasks are closures and may borrow from the +//! enclosing scope (`'env`); the scope blocks until every one has finished, which is what makes +//! those borrows sound. +//! +//! Use [`scope_unbounded`](crate::scope_unbounded::scope_unbounded) instead when a running job can +//! discover more work, so the total isn't known up front. use std::{ any::Any, @@ -238,8 +246,8 @@ impl<'scope, 'env: 'scope, R: Send + 'env> Drop for Scope<'scope, 'env, R> { /// Be aware that although this function avoids starving other independently spawned tasks, any /// other code running concurrently in the same task will be suspended during the call to /// block_in_place. This can happen e.g. when using the `join!` macro. To avoid this issue, call -/// `scope_and_block` in `spawn_blocking`. -pub fn scope_and_block<'env, F, R>(number_of_tasks: usize, f: F) -> impl Iterator +/// `scope_bounded` in `spawn_blocking`. +pub fn scope_bounded<'env, F, R>(number_of_tasks: usize, f: F) -> impl Iterator where R: Send + 'env, F: for<'scope> FnOnce(&'scope Scope<'scope, 'env, R>) + 'env, @@ -266,7 +274,10 @@ where #[cfg(test)] mod tests { - use std::panic::{AssertUnwindSafe, catch_unwind}; + use std::{ + panic::{AssertUnwindSafe, catch_unwind}, + sync::atomic::AtomicUsize, + }; use super::*; @@ -300,7 +311,7 @@ mod tests { let started = Instant::now(); let results = tokio::task::spawn_blocking(move || { - scope_and_block(JOBS, |scope| { + scope_bounded(JOBS, |scope| { for i in 0..JOBS { scope.spawn(move || i); } @@ -317,7 +328,7 @@ mod tests { }); assert!( elapsed < RELEASE_AFTER / 2, - "scope_and_block took {elapsed:?}; it should not depend on an occupied worker thread \ + "scope_bounded took {elapsed:?}; it should not depend on an occupied worker thread \ freeing up" ); @@ -331,7 +342,7 @@ mod tests { #[tokio::test(flavor = "current_thread")] async fn test_scope_current_thread_runtime() { let results = tokio::task::spawn_blocking(|| { - scope_and_block(16, |scope| { + scope_bounded(16, |scope| { for i in 0..16 { scope.spawn(move || i); } @@ -354,7 +365,7 @@ mod tests { const PER_JOB: Duration = Duration::from_millis(50); let started = Instant::now(); let results = tokio::task::spawn_blocking(|| { - scope_and_block(JOBS, |scope| { + scope_bounded(JOBS, |scope| { for i in 0..JOBS { scope.spawn(move || { thread::sleep(PER_JOB); @@ -372,13 +383,13 @@ mod tests { // so a slow machine won't make this flaky. assert!( elapsed < (JOBS as u32 * PER_JOB) / 2, - "scope_and_block took {elapsed:?}; expected parallel speedup across worker threads" + "scope_bounded took {elapsed:?}; expected parallel speedup across worker threads" ); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_scope() { - let results = scope_and_block(1000, |scope| { + let results = scope_bounded(1000, |scope| { for i in 0..1000 { scope.spawn(move || i); } @@ -392,7 +403,7 @@ mod tests { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_empty_scope() { - let results = scope_and_block(0, |scope| { + let results = scope_bounded(0, |scope| { if false { scope.spawn(|| 42); } @@ -402,7 +413,7 @@ mod tests { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_single_task() { - let results = scope_and_block(1, |scope| { + let results = scope_bounded(1, |scope| { scope.spawn(|| 42); }) .collect::>(); @@ -411,7 +422,7 @@ mod tests { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_task_finish_before_scope() { - let results = scope_and_block(1, |scope| { + let results = scope_bounded(1, |scope| { scope.spawn(|| 42); thread::sleep(std::time::Duration::from_millis(100)); }) @@ -421,7 +432,7 @@ mod tests { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_task_finish_after_scope() { - let results = scope_and_block(1, |scope| { + let results = scope_bounded(1, |scope| { scope.spawn(|| { thread::sleep(std::time::Duration::from_millis(100)); 42 @@ -434,7 +445,7 @@ mod tests { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_panic_in_scope_factory() { let result = catch_unwind(AssertUnwindSafe(|| { - let _results = scope_and_block(1000, |scope| { + let _results = scope_bounded(1000, |scope| { for i in 0..500 { scope.spawn(move || i); } @@ -452,7 +463,7 @@ mod tests { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn test_panic_in_scope_task() { let result = catch_unwind(AssertUnwindSafe(|| { - let _results = scope_and_block(1000, |scope| { + let _results = scope_bounded(1000, |scope| { for i in 0..1000 { scope.spawn(move || { if i == 500 { diff --git a/turbopack/crates/turbo-tasks/src/scope_unbounded.rs b/turbopack/crates/turbo-tasks/src/scope_unbounded.rs new file mode 100644 index 000000000000..a88f2e2d0199 --- /dev/null +++ b/turbopack/crates/turbo-tasks/src/scope_unbounded.rs @@ -0,0 +1,977 @@ +//! Unbounded scoped parallelism: enables running jobs that can discover and enqueue more work + +use std::{ + any::Any, + ops::ControlFlow, + panic::{self, AssertUnwindSafe, catch_unwind}, + sync::{ + Arc, OnceLock, SyncView, + atomic::{AtomicBool, AtomicUsize, Ordering}, + mpmc, + }, + time::Duration, +}; + +use fixedbitset::FixedBitSet; +use parking_lot::{Condvar, Mutex, RwLock}; +use tokio::{runtime::Handle, task::AbortHandle}; +use tracing::{Span, info_span}; + +use crate::{TurboTasksApi, manager::try_turbo_tasks, turbo_tasks_scope}; + +/// How long a scope worker waits on an empty queue before exiting. +/// +/// Optimizes respawning which triggers overhead managing WorkerSlots and the accumulator variables +/// pausing for a short time is worthwhile to avoid that. +const WORKER_IDLE_TIMEOUT: Duration = Duration::from_micros(100); + +/// Runs `run` over `initial` and everything it transitively spawns, returning once every item has +/// been processed. No results are collected; jobs communicate through state captured in `run`. Use +/// [`scope_unbounded_with`] to accumulate a value instead. +////// +/// Items must be `'static` (they sit in a queue drained by other threads); the `run` closure may +/// borrow `'env` data. +/// +/// # Aborting +/// +/// Both [`ControlFlow::Break`] and a panic abandon all queued-but-unstarted items, so the scope +/// returns as soon as the currently-running jobs finish. Jobs already in flight on other threads +/// are **not** interrupted in either case. +pub fn scope_unbounded<'env, T, F>(initial: impl IntoIterator, run: F) +where + T: Send + 'static, + F: Fn(&Scope<'_, T, ()>, T) -> ControlFlow<()> + Send + Sync + 'env, +{ + scope_unbounded_with( + initial, + || (), + |spawner, item, ()| run(spawner, item), + |(), ()| (), + ) +} + +/// [`scope_unbounded`], plus a per-drainer accumulator folded into a single return value. +/// +/// Each drainer builds its own accumulator with `init`, `run` mutates it in place while processing +/// items, and the accumulators are combined pairwise with `merge` as drainers finish. `merge` must +/// be associative and commutative — drainers finish in a nondeterministic order, so the grouping +/// and ordering of the folds are not specified. +/// +/// This exists so `run` can efficiently accumulate with minimal locking overhead managed by the +/// scope. +/// +/// `init` is called potentially many times for each thread context. +/// +/// Returns `init()` when no item is ever processed (e.g. an empty `initial`). +/// +/// # Panics and aborts +/// +/// Both [`ControlFlow::Break`] and a panic abort the scope, abandoning every queued-but-unstarted +/// item (see [`scope_unbounded`]). They differ in what comes back: +/// +/// - On `Break`, results accumulated before the abort are returned as usual; the abandoned items +/// simply never contributed. +/// - On a panic, the panic is re-raised after the join and **all accumulated results are +/// discarded** — the return value is only produced on the normal path. +pub fn scope_unbounded_with<'env, T, R, F, Init, Merge>( + initial: impl IntoIterator, + init: Init, + run: F, + merge: Merge, +) -> R +where + T: Send + 'static, + R: Send + 'env, + F: Fn(&Scope<'_, T, R>, T, &mut R) -> ControlFlow<()> + Send + Sync + 'env, + Init: Fn() -> R + Send + Sync + 'env, + Merge: Fn(R, R) -> R + Send + Sync + 'env, +{ + let handle = Handle::current(); + // One worker per runtime thread beyond the calling thread + let max_workers = handle.metrics().num_workers().saturating_sub(1); + let span = Span::current(); + + // `ScopeInner` is parameterized over the borrow lifetime, so these go in as ordinary + // references. The one erasure to `'static` is at the tokio hand-off in + // `spawn_worker_if_needed`. + let init_ref: &(dyn Fn() -> R + Send + Sync + '_) = &init; + let merge_ref: &(dyn Fn(R, R) -> R + Send + Sync + '_) = &merge; + + let (sender, receiver) = mpmc::channel(); + let mut inner = ScopeInner { + remaining_tasks: AtomicUsize::new(0), + panic: OnceLock::new(), + work_queue: receiver, + work_queue_sender: RwLock::new(Some(sender)), + aborted: AtomicBool::new(false), + available_slots: AtomicUsize::new(max_workers), + workers: Mutex::new(WorkerSlots::new(max_workers)), + handle: handle.clone(), + span: span.clone(), + workers_idle: Condvar::new(), + turbo_tasks: try_turbo_tasks(), + run: &run, + results: Mutex::new(None), + init: init_ref, + merge: merge_ref, + }; + + // Arm the join guard before anything can spawn, so a worker can never outlive the join. + let joiner = Joiner { inner: &inner }; + + // Increment remaining tasks to ensure the scope cannot exit before all tasks are enqueued + inner.remaining_tasks.fetch_add(1, Ordering::Relaxed); + for item in initial { + enqueue(&inner, item); + } + + // Drain and join before checking for a panic. Every drainer has merged its accumulator by the + // time this returns. + drop(joiner); + + if let Some(err) = inner.panic.take() { + panic::resume_unwind(err.into_inner()); + } + + inner.results.lock().take().unwrap_or_else(init) +} + +/// Handle passed to the `run` closure of [`scope_unbounded`], used to enqueue additional items into +/// the same scope. +pub struct Scope<'scope, T: Send + 'static, R = ()> { + inner: &'scope ScopeInner<'scope, T, R>, +} + +impl Scope<'_, T, R> { + /// Enqueue another item to be processed by `run`. Callable any number of times from inside + /// `run`, on any drainer thread. + /// + /// Silently drops `item` once the scope has aborted. + pub fn spawn(&self, item: T) { + enqueue(self.inner, item); + } +} + +/// A reference to the shared per-item closure for a [`scope_unbounded`] run. `'run` is the lifetime +/// of the borrows it captures (`'env` at the call site, erased to `'static` when handed to tokio). +/// `R` is the per-drainer accumulator threaded through by [`scope_unbounded_with`]. +type RunFn<'run, T, R> = + &'run (dyn Fn(&Scope<'_, T, R>, T, &mut R) -> ControlFlow<()> + Send + Sync + 'run); + +/// Shared state for a [`scope_unbounded`] run, living on the caller's stack. +/// +/// `'run` is the lifetime of the borrows held by the `run`/`init`/`merge` closures (`'env` at the +/// call site). It stays a real lifetime here rather than being pinned to `'static` so the fields +/// don't each force `R: 'static`; the single erasure to `'static` happens at the [`Drainable`] +/// hand-off to tokio. +struct ScopeInner<'run, T: Send + 'static, R> { + /// Items enqueued but not yet finished. The scope is done exactly when this reaches zero; see + /// [`enqueue`] for the increment-before-push ordering that makes zero reliable. + remaining_tasks: AtomicUsize, + /// First panic raised while processing an item; propagated to the caller after the join. + panic: OnceLock>>, + /// Receiving end of the work queue, shared by every drainer. + work_queue: mpmc::Receiver, + /// Sending end of the queue, modeled so we can `take` and thus close the queue + work_queue_sender: RwLock>>, + aborted: AtomicBool, + /// Spawn budget, mutated under the workers slot, atomic so it can be read outside of it. + available_slots: AtomicUsize, + workers: Mutex, + /// Triggered when the last worker in `workers` is cleared. + workers_idle: Condvar, + handle: Handle, + span: Span, + turbo_tasks: Option>, + /// The per-item closure. + run: RunFn<'run, T, R>, + init: &'run (dyn Fn() -> R + Send + Sync + 'run), + merge: &'run (dyn Fn(R, R) -> R + Send + Sync + 'run), + // Accumulated results, workers aggregate into this using init/merge as workers exit their + // scope + results: Mutex>, +} + +impl ScopeInner<'_, T, R> { + /// Closes the work queue by dropping the only sender. Every blocked `recv` returns `Err` once + /// this runs and the buffer is drained, which is how drainers learn the scope is finished. + /// Idempotent. + fn close(&self) { + drop(self.work_queue_sender.write().take()); + } + + /// Abandons all queued-but-unstarted work. Idempotent. + fn abort(&self) { + self.aborted.store(true, Ordering::Release); + self.close(); + } + + fn on_item_finished(&self) { + if self.remaining_tasks.fetch_sub(1, Ordering::Release) == 1 { + self.close(); + } + } + + /// Keeps the first panic seen; later ones are dropped. + fn record_panic(&self, err: Box) { + self.abort(); + let _ = self.panic.set(SyncView::new(err)); + } + + /// Drain loop, run by both the workers and the calling thread until the scope terminates. + /// + /// - A scope worker (`is_worker`) exits after [`WORKER_IDLE_TIMEOUT`] on an empty queue + /// - The calling thread blocks until the queue closes, which requires every item to be + /// finished. + fn drain(&self, is_worker: bool) { + if is_worker && let Some(turbo_tasks) = &self.turbo_tasks { + turbo_tasks_scope(turbo_tasks.clone(), || self.drain_loop(is_worker)) + } else { + self.drain_loop(is_worker) + } + } + + fn drain_loop(&self, is_worker: bool) { + let mut acc: Option = None; + while let Some(item) = if is_worker { + self.work_queue.recv_timeout(WORKER_IDLE_TIMEOUT).ok() + } else { + self.work_queue.recv().ok() + } { + // Post-abort: discard without running, so the wind-down can't re-grow the queue. + if self.aborted.load(Ordering::Acquire) { + self.on_item_finished(); + continue; + } + let spawner = Scope { inner: self }; + let result = catch_unwind(AssertUnwindSafe(|| { + // Lazily init the thread local accumulator only when we are going to execute an + // item + let acc = acc.get_or_insert_with(self.init); + (self.run)(&spawner, item, acc) + })); + + match result { + Ok(ControlFlow::Continue(())) => {} + Ok(ControlFlow::Break(())) => { + self.abort(); + } + // A panic aborts too; see `scope_unbounded`. + Err(panic) => { + self.record_panic(panic); + } + }; + self.on_item_finished(); + } + + // Fold this drainer's accumulator into the shared results slot + if let Some(acc) = acc { + let merged = catch_unwind(AssertUnwindSafe(|| { + let mut results = self.results.lock(); + *results = Some(match results.take() { + Some(existing) => (self.merge)(existing, acc), + None => acc, + }); + })); + if let Err(panic) = merged { + self.record_panic(panic); + } + } + } +} + +/// Account for and enqueue one item. The increment must happen before the push: pushing first would +/// let another drainer pop and finish the item before it is counted, so `remaining_tasks` could hit +/// zero with work still live. +fn enqueue(inner: &ScopeInner<'_, T, R>, item: T) { + if inner.aborted.load(Ordering::Acquire) { + return; + } + let num_tasks = inner.remaining_tasks.fetch_add(1, Ordering::Relaxed) + 1; + let sent = { + let sender = inner.work_queue_sender.read(); + match sender.as_ref() { + Some(sender) => sender.send(item).is_ok(), + // Closed: the scope is winding down (aborted, or already finished). + None => false, + } + }; + if !sent { + inner.on_item_finished(); // since the item won't execute decrement now + return; + } + spawn_worker_if_needed(inner, num_tasks); +} + +/// Re-arm one worker if the scope is running below its budget. +fn spawn_worker_if_needed( + inner: &ScopeInner<'_, T, R>, + num_enqueued_tasks: usize, +) { + if num_enqueued_tasks <= 1 + || inner.available_slots.load(Ordering::Relaxed) == 0 + || inner.aborted.load(Ordering::Acquire) + { + return; + } + + // SAFETY: `Joiner::drop` waits for every worker slot to be released before returning, and the + // slot for this worker is claimed below under the table lock *before* the spawn, so no erased + // reference can outlive `'env` or the `inner` stack slot. + let erased: &(dyn Drainable + Send + Sync + '_) = inner; + let erased: &'static (dyn Drainable + Send + Sync + 'static) = unsafe { + std::mem::transmute::< + &(dyn Drainable + Send + Sync + '_), + &'static (dyn Drainable + Send + Sync + 'static), + >(erased) + }; + + let mut slots = inner.workers.lock(); + let Some(slot) = slots.free_slot() else { + return; + }; + let span = inner.span.clone(); + // capture before the spawn and move into it + let guard = erased.claim_worker_slot(slot); + let handle = inner + .handle + .spawn(async move { + let _span = span.entered(); + let _guard = guard; + erased.drain(true); + }) + .abort_handle(); + slots.occupy(slot, handle, &inner.available_slots); +} + +/// The drain loop with the accumulator type erased. +/// +/// Worker tasks are spawned onto tokio and so must be `'static`, but the accumulator `R` borrows +/// `'env`. A worker only ever needs to *run* the loop — it never names an `R` — so it holds the +/// scope through this trait instead of the concrete [`ScopeInner`], keeping `R` out of the spawned +/// future's type entirely. +trait Drainable { + fn drain(&self, is_worker: bool); + /// Take ownership of `slot`'s release, which [`spawn_worker_if_needed`] has already claimed. + /// On the trait so a spawned worker can build its guard without naming `R`. + fn claim_worker_slot(&self, slot: usize) -> WorkerGuard<'_>; +} + +impl Drainable for ScopeInner<'_, T, R> { + fn drain(&self, is_worker: bool) { + ScopeInner::drain(self, is_worker) + } + + fn claim_worker_slot(&self, slot: usize) -> WorkerGuard<'_> { + WorkerGuard { + slots: &self.workers, + workers_idle: &self.workers_idle, + available_slots: &self.available_slots, + slot, + } + } +} + +/// Fixed table of worker slots, indexed by slot number, with a bitset of which are occupied. +/// +/// This is both the spawn budget and the join set, because a slot is occupied from *before* its +/// task exists until *after* that task's last access to the caller's frame: +struct WorkerSlots { + handles: Vec>, + /// Bit `i` set means slot `i` is occupied — claimed by [`Self::occupy`] and not yet released + /// by a [`WorkerGuard`]. Tracks *occupancy*, not handle presence. + occupied: FixedBitSet, +} + +impl WorkerSlots { + fn new(max_workers: usize) -> Self { + Self { + handles: vec![None; max_workers], + occupied: FixedBitSet::with_capacity(max_workers), + } + } + + fn free_slot(&self) -> Option { + self.occupied.zeroes().next() + } + + /// Record a newly spawned worker in `slot`, which must have come from [`Self::free_slot`]. + fn occupy(&mut self, slot: usize, handle: AbortHandle, available_slots: &AtomicUsize) { + debug_assert!( + !self.occupied.contains(slot), + "slot {slot} already occupied" + ); + available_slots.fetch_sub(1, Ordering::Relaxed); + self.occupied.insert(slot); + let previous = self.handles[slot].replace(handle); + debug_assert!(previous.is_none(), "slot {slot} held a live handle"); + } + + /// Dropping the handle here keeps the finished task's allocation from outliving the worker. + fn release(&mut self, slot: usize, available_slots: &AtomicUsize) { + available_slots.fetch_add(1, Ordering::Relaxed); + self.occupied.remove(slot); + self.handles[slot] = None; + } + + /// Whether every slot is free, i.e. no task can still touch the caller's frame. The predicate + /// [`Joiner::drop`] waits on. + fn is_idle(&self) -> bool { + self.occupied.is_clear() + } +} + +/// Releases a worker's slot and wakes [`Joiner::drop`] when it is the last one. +struct WorkerGuard<'a> { + slots: &'a Mutex, + workers_idle: &'a Condvar, + available_slots: &'a AtomicUsize, + slot: usize, +} + +impl Drop for WorkerGuard<'_> { + fn drop(&mut self) { + let mut slots_guard = self.slots.lock(); + slots_guard.release(self.slot, self.available_slots); + if slots_guard.is_idle() { + // Still holding the lock: the predicate the joiner waits on is read under this same + // lock, so it cannot go true between its check and its `wait`. + self.workers_idle.notify_all(); + } + } +} + +/// Drains the queue and joins the workers, on the return path and on an unwind alike. +struct Joiner<'a, 'run, T: Send + 'static, R> { + inner: &'a ScopeInner<'run, T, R>, +} + +impl Drop for Joiner<'_, '_, T, R> { + fn drop(&mut self) { + // Discharge the placeholder item that covered the seeding loop. + self.inner.on_item_finished(); + // Returns only once the queue is closed, so no new work can arrive after this. + self.inner.drain(false); + // The queue is now closed so no workers can spawn. Capture all the handles. + // There should be no contention on this slot + let _span = info_span!("blocking: waiting for scope to end").entered(); + let handles: Vec<_> = { + let mut slots = self.inner.workers.lock(); + slots.handles.iter_mut().filter_map(Option::take).collect() + }; + + // Abort all workers, that way workers we have spawned but have never run get dropped and + // release their slots. Otherwise a contended runtime could delay shutdown of the scope. + for handle in handles { + handle.abort(); + } + + // Wait for all workers to exit and drop their guards. This should be fast, after the + // aborts and the queue is dropped each worker just needs to merge their accumulated + // stats. So we wait for that. + let mut slots = self.inner.workers.lock(); + while !slots.is_idle() { + self.inner.workers_idle.wait(&mut slots); + } + } +} +#[cfg(test)] +mod tests { + use std::{ + sync::{Arc, atomic::AtomicUsize}, + thread, + time::Duration, + }; + + use super::*; + + /// Runs `body` on a runtime with the I/O driver disabled, so the test also works under Miri. + /// + /// `#[tokio::test]` hardcodes `Builder::enable_all()`, whose I/O driver calls + /// `kqueue()`/`epoll_create1()` — Miri has no shim for either, so such a test aborts at runtime + /// construction before reaching any code under test. Nothing here needs I/O. + /// + /// `body` runs on a blocking thread, as production callers are expected to: `scope_unbounded` + /// blocks, and its `block_in_place` requires a multi-thread runtime. + fn with_runtime(worker_threads: usize, body: F) -> T + where + F: FnOnce() -> T + Send + 'static, + T: Send + 'static, + { + let runtime = tokio::runtime::Builder::new_multi_thread() + .worker_threads(worker_threads) + .enable_time() + .build() + .unwrap(); + runtime.block_on(async { tokio::task::spawn_blocking(body).await.unwrap() }) + } + + /// A single `run` call enqueues a large batch of leaves; every one must be processed. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_wide_burst_of_leaves() { + const CHILDREN: usize = 1000; + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + tokio::task::spawn_blocking(move || { + scope_unbounded(std::iter::once(0usize), move |spawner, item| { + processed_clone.fetch_add(1, Ordering::SeqCst); + if item == 0 { + // The root fans out to CHILDREN leaves. + for i in 0..CHILDREN { + spawner.spawn(1 + i); + } + } + ControlFlow::Continue(()) + }); + }) + .await + .unwrap(); + // 1 root + CHILDREN leaves. + assert_eq!(processed.load(Ordering::SeqCst), 1 + CHILDREN); + } + + /// A slow seeding iterator must not let the scope finish early. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_slow_seeding_iterator_completes() { + const SEEDS: usize = 16; + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + tokio::task::spawn_blocking(move || { + // Each `next()` blocks briefly, so the queue drains to empty before the next seed + // arrives. + let slow_seeds = std::iter::from_fn({ + let mut next = 0; + move || { + if next == SEEDS { + return None; + } + thread::sleep(Duration::from_millis(2)); + next += 1; + Some(next - 1) + } + }); + scope_unbounded(slow_seeds, move |_spawner, _item| { + processed_clone.fetch_add(1, Ordering::SeqCst); + ControlFlow::Continue(()) + }); + }) + .await + .unwrap(); + assert_eq!( + processed.load(Ordering::SeqCst), + SEEDS, + "seeds produced after the queue briefly drained must still be processed" + ); + } + + /// Aborting in the middle of a deep, still-growing cascade must terminate rather than hang: + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_abort_during_cascade() { + // Each item spawns two children until the id exceeds the bound, so the queue is still + // growing when the abort lands. + const MAX_ID: usize = 1 << 14; + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + tokio::task::spawn_blocking(move || { + scope_unbounded(std::iter::once(1usize), move |spawner, id| { + let n = processed_clone.fetch_add(1, Ordering::SeqCst); + if n == 100 { + return ControlFlow::Break(()); + } + let (left, right) = (id * 2, id * 2 + 1); + if left <= MAX_ID { + spawner.spawn(left); + } + if right <= MAX_ID { + spawner.spawn(right); + } + ControlFlow::Continue(()) + }); + }) + .await + .unwrap(); + let count = processed.load(Ordering::SeqCst); + assert!( + count < MAX_ID, + "abort must cut the cascade short, but {count} items ran" + ); + } + + /// `spawn` issued *after* the abort has latched must be dropped, not enqueued — the case a job + /// finishing concurrently with another job's abort hits. A `spawn` that counted an item into + /// `remaining_tasks` without queueing it would never reach zero, so this hangs rather than + /// fails. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_unbounded_spawn_after_abort_is_dropped() { + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + const SEEDS: usize = 64; + tokio::task::spawn_blocking(move || { + scope_unbounded(0..SEEDS, move |spawner, item| { + processed_clone.fetch_add(1, Ordering::SeqCst); + // Spawning *before* the `Break` is the point: these spawns race the abort latch and + // must be dropped rather than counted-but-unqueued. + for i in 0..1000 { + spawner.spawn(SEEDS + item * 1000 + i); + } + ControlFlow::Break(()) + }); + }) + .await + .unwrap(); + // Only seeds may run: every spawned id is >= SEEDS, so processing even one would push the + // count past the seed total. + let count = processed.load(Ordering::SeqCst); + assert!( + count <= SEEDS, + "post-abort spawns must be dropped, but {count} items ran" + ); + } + + /// Abort on a `current_thread` runtime, where the calling thread is the only drainer. + #[tokio::test(flavor = "current_thread")] + async fn test_unbounded_abort_current_thread_runtime() { + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + tokio::task::spawn_blocking(move || { + scope_unbounded(0..1000usize, move |spawner, _item| { + processed_clone.fetch_add(1, Ordering::SeqCst); + spawner.spawn(9999); + ControlFlow::Break(()) + }); + }) + .await + .unwrap(); + // With a single drainer the abort lands before any other item is picked up. + assert_eq!(processed.load(Ordering::SeqCst), 1); + } + + /// A panic that happens while the scope is aborting still propagates rather than being + /// swallowed by the wind-down: the abort's queue-clear races the panic's unwind through + /// `catch_unwind` -> `on_item_finished`. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_unbounded_abort_then_panic() { + let result = catch_unwind(AssertUnwindSafe(|| { + scope_unbounded(0..1000usize, |_spawner, item| { + if item == 0 { + panic!("Intentional panic"); + } + ControlFlow::Break(()) + }); + unreachable!(); + })); + let err = result.expect_err("the panic must propagate even though the scope aborted"); + assert_eq!(err.downcast_ref::<&str>(), Some(&"Intentional panic")); + } + + /// A panic in a `run` invocation propagates after all in-flight work is joined, and aborts the + /// scope: the queued-but-unstarted items are abandoned rather than run. + /// + /// The first item panics, so with a large seed set almost nothing else should be dispatched. + /// Items already picked up by another drainer still complete, so the bound is "far fewer than + /// seeded" rather than exactly one. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_panic_propagates_and_abandons_queue() { + const ITEMS: usize = 10_000; + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + let result = catch_unwind(AssertUnwindSafe(|| { + scope_unbounded(0..ITEMS, move |_spawner, item| { + processed_clone.fetch_add(1, Ordering::SeqCst); + if item == 0 { + panic!("Intentional panic"); + } + ControlFlow::Continue(()) + }); + unreachable!(); + })); + let err = result.expect_err("the panic must propagate"); + assert_eq!(err.downcast_ref::<&str>(), Some(&"Intentional panic")); + let count = processed.load(Ordering::SeqCst); + assert!( + count < ITEMS, + "a panic must abandon the queue, but all {ITEMS} items ran" + ); + } + + // ----------------------------------------------------------------------- + // scope_unbounded_with (fold results) + // ----------------------------------------------------------------------- + + /// The accumulator must be per-drainer, not shared: collecting into a `Vec` and merging by + /// concatenation must preserve every element even with several drainers running. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_with_collects_all_values() { + const ITEMS: usize = 500; + let mut collected = tokio::task::spawn_blocking(|| { + scope_unbounded_with( + 0..ITEMS, + Vec::new, + |_spawner, item: usize, acc: &mut Vec| { + acc.push(item); + ControlFlow::Continue(()) + }, + |mut a: Vec, b| { + a.extend(b); + a + }, + ) + }) + .await + .unwrap(); + collected.sort_unstable(); + assert_eq!(collected, (0..ITEMS).collect::>()); + } + + /// With no items, no drainer builds an accumulator, so the result is exactly one `init()` — + /// not a fold of one per drainer that happened to start. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_unbounded_with_empty_returns_init() { + let total = tokio::task::spawn_blocking(|| { + scope_unbounded_with( + std::iter::empty::(), + || 42usize, + |_spawner, _item, _acc| ControlFlow::Continue(()), + |a, b| a + b, + ) + }) + .await + .unwrap(); + assert_eq!(total, 42, "expected exactly one init(), got {total}"); + } + + /// A scope driven **directly** on a `current_thread` runtime — not via `spawn_blocking` — must + /// still complete. + #[tokio::test(flavor = "current_thread")] + async fn test_unbounded_current_thread_direct_call_completes() { + let processed = Arc::new(AtomicUsize::new(0)); + let processed_clone = processed.clone(); + scope_unbounded(0..8usize, move |spawner, item| { + processed_clone.fetch_add(1, Ordering::SeqCst); + if item < 3 { + spawner.spawn(100 + item); + } + ControlFlow::Continue(()) + }); + assert_eq!(processed.load(Ordering::SeqCst), 11); + } + + /// Aborting returns the results accumulated up to that point rather than discarding them — + /// only the abandoned items are missing. The run must still terminate cleanly. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_with_abort_returns_partial_results() { + let processed = tokio::task::spawn_blocking(|| { + scope_unbounded_with( + 0..1000usize, + || 0usize, + |_spawner, item, acc| { + *acc += 1; + if item == 0 { + return ControlFlow::Break(()); + } + ControlFlow::Continue(()) + }, + |a, b| a + b, + ) + }) + .await + .unwrap(); + // At least the aborting item ran, and the abort must have cut the run short. + assert!(processed >= 1, "expected the aborting item to be counted"); + assert!( + processed < 1000, + "abort should abandon queued items, but all {processed} ran" + ); + } + + /// A panic must propagate through the fold path without deadlocking the join, which drainers + /// reach only after their merge. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_unbounded_with_panic_propagates() { + let result = catch_unwind(AssertUnwindSafe(|| { + scope_unbounded_with( + 0..100usize, + || 0usize, + |_spawner, item, acc| { + if item == 50 { + panic!("Intentional panic"); + } + *acc += 1; + ControlFlow::Continue(()) + }, + |a, b| a + b, + ); + unreachable!(); + })); + let err = result.expect_err("the panic must propagate out of the fold API"); + assert_eq!(err.downcast_ref::<&str>(), Some(&"Intentional panic")); + } + + /// The accumulator may borrow `'env` data (it is not `'static`), mirroring how `run` may. + #[test] + fn test_unbounded_with_borrowed_accumulator() { + let label = String::from("item"); + let count = with_runtime(4, move || { + let label = &label; + scope_unbounded_with( + 0..32usize, + Vec::new, + |_spawner, item: usize, acc: &mut Vec| { + acc.push(format!("{label}-{item}")); + ControlFlow::Continue(()) + }, + |mut a: Vec, b| { + a.extend(b); + a + }, + ) + .len() + }); + assert_eq!(count, 32); + } + + // ----------------------------------------------------------------------- + // worker exit / respawn tests + // + // `init` runs once per drainer that receives an item, so counting `init` calls counts *distinct + // drainer lifetimes* — the only externally visible signal that a worker exited and a later one + // replaced it. + // ----------------------------------------------------------------------- + + /// A worker exits once the queue is empty, and a later `spawn` re-arms one: the scope must + /// still finish work produced after every worker has gone away. + /// + /// Serialized by construction — one item in flight at a time, with a gap long enough that any + /// worker has certainly timed out — so reaching the second item at all exercises the respawn + /// path rather than a still-live worker. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_worker_respawns_after_going_idle() { + let inits = Arc::new(AtomicUsize::new(0)); + let counted = inits.clone(); + let processed = Arc::new(AtomicUsize::new(0)); + let ran = processed.clone(); + tokio::task::spawn_blocking(move || { + scope_unbounded_with( + std::iter::once(0usize), + move || { + counted.fetch_add(1, Ordering::SeqCst); + }, + move |spawner, item, ()| { + ran.fetch_add(1, Ordering::SeqCst); + if item == 0 { + // Let the queue sit empty long enough that any worker has exited, then + // produce work again. A fresh drainer is the only thing that can pick it + // up. + thread::sleep(Duration::from_millis(100)); + spawner.spawn(1); + } + ControlFlow::Continue(()) + }, + |(), ()| (), + ) + }) + .await + .unwrap(); + assert_eq!( + processed.load(Ordering::SeqCst), + 2, + "work spawned after the pool went idle must still run" + ); + // Both items ran (the scope only returns once the queue is drained), and at least one + // drainer built an accumulator. The exact count depends on which drainer wins each item. + let count = inits.load(Ordering::SeqCst); + assert!( + (1..=2).contains(&count), + "expected 1-2 drainer lifetimes, got {count}" + ); + } + + /// A scope that never has queued work must not occupy a worker at all. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_empty_spawns_no_workers() { + let inits = Arc::new(AtomicUsize::new(0)); + let counted = inits.clone(); + tokio::task::spawn_blocking(move || { + scope_unbounded_with( + std::iter::empty::(), + move || counted.fetch_add(1, Ordering::SeqCst), + |_spawner, _item, _acc| ControlFlow::Continue(()), + |a, b| a + b, + ) + }) + .await + .unwrap(); + assert_eq!( + inits.load(Ordering::SeqCst), + 1, + "expected only the terminal identity init(), not a per-drainer one" + ); + } + + /// Sustained work keeps workers alive rather than churning them: + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn test_unbounded_busy_queue_does_not_churn_workers() { + const ITEMS: usize = 20_000; + let inits = Arc::new(AtomicUsize::new(0)); + let counted = inits.clone(); + let processed = tokio::task::spawn_blocking(move || { + scope_unbounded_with( + 0..ITEMS, + move || { + counted.fetch_add(1, Ordering::SeqCst); + 0usize + }, + |_spawner, _item, acc| { + *acc += 1; + ControlFlow::Continue(()) + }, + |a, b| a + b, + ) + }) + .await + .unwrap(); + assert_eq!(processed, ITEMS, "every item must run"); + // 4 runtime workers => 3 scope workers + the calling thread at any instant. The bound is + // loose because a worker can still lose a race to the last queued item and be re-armed, and + // the rest of the suite competes for the same threads — but a churning implementation lands + // in the thousands, so anything near the worker count proves the timeout is doing its job. + let count = inits.load(Ordering::SeqCst); + assert!( + count <= 16, + "a saturated queue should not churn drainers, got {count} lifetimes for {ITEMS} items" + ); + } + + /// The join must not depend on tokio scheduling, even when every runtime thread is contended + /// and workers are still mid-drain. + #[test] + fn test_unbounded_join_under_thread_starvation() { + let runtime = tokio::runtime::Builder::new_multi_thread() + .worker_threads(2) + .enable_time() + .build() + .unwrap(); + runtime.block_on(async { + let mut scopes = Vec::new(); + for _ in 0..8 { + scopes.push(tokio::task::spawn_blocking(|| { + let processed = Arc::new(AtomicUsize::new(0)); + let counted = processed.clone(); + scope_unbounded(0..200usize, move |spawner, item| { + counted.fetch_add(1, Ordering::SeqCst); + // Keep the queue growing so workers are still draining at join time. + if item < 200 { + spawner.spawn(1000 + item); + } + thread::yield_now(); + ControlFlow::Continue(()) + }); + processed.load(Ordering::SeqCst) + })); + } + for scope in scopes { + assert_eq!(scope.await.unwrap(), 400, "every scope must drain fully"); + } + }); + } +} From 9fbec9300ff10ed5f6982488adb0a191062fa0ac Mon Sep 17 00:00:00 2001 From: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com> Date: Fri, 21 Aug 2026 20:25:27 +0200 Subject: [PATCH 5/7] Migrate some async blocks to async closures (#97666) This is everything except `turbo-tasks-[backend]` Using ast-grep replacement: - search: `|$$$ARGS| async move {$$$BODY}` - replace: `async |$$$ARGS| {$$$BODY}` This also made it obvious in some places that we don't actually need async-await, e.g. `.map(async |v| v.to_resolved.await)` --- crates/next-api/src/aggregate_hmr.rs | 8 ++-- crates/next-api/src/app.rs | 2 +- crates/next-api/src/client_references.rs | 2 +- crates/next-api/src/dynamic_imports.rs | 4 +- crates/next-api/src/module_graph.rs | 6 +-- crates/next-api/src/versioned_content_map.rs | 4 +- .../next_app/app_client_references_chunks.rs | 6 +-- crates/next-core/src/pages_structure.rs | 8 ++-- crates/next-core/src/segment_config.rs | 4 +- .../src/next_api/project.rs | 2 +- .../turbopack-analyze/tests/split_chunk.rs | 2 +- turbopack/crates/turbopack-bench/src/lib.rs | 8 ++-- .../crates/turbopack-bench/src/util/mod.rs | 4 +- .../crates/turbopack-cli/src/build/mod.rs | 38 ++++++++----------- .../turbopack-cli/src/dev/web_entry_source.rs | 2 +- .../crates/turbopack-core/src/issue/mod.rs | 2 +- .../turbopack-core/src/module_graph/mod.rs | 8 ++-- .../turbopack-core/src/reference/mod.rs | 8 ++-- .../crates/turbopack-core/src/resolve/mod.rs | 12 +++--- .../turbopack-core/src/resolve/parse.rs | 8 ++-- .../crates/turbopack-css/src/chunk/mod.rs | 8 ++-- .../crates/turbopack-dev-server/src/html.rs | 2 +- .../src/introspect/mod.rs | 2 +- .../src/source/asset_graph.rs | 10 ++--- .../src/source/combined.rs | 10 ++--- .../src/source/route_tree.rs | 2 +- .../turbopack-dev-server/src/source/router.rs | 2 +- .../src/source/static_assets.rs | 2 +- .../src/source/wrapping_source.rs | 2 +- .../benches/references.rs | 2 +- .../src/chunk/code_module_ids_and_paths.rs | 2 +- .../src/chunk/content_entry.rs | 2 +- .../turbopack-ecmascript/src/chunk/mod.rs | 2 +- .../src/chunk/placeable.rs | 2 +- .../src/chunk_list/version.rs | 2 +- .../turbopack-ecmascript/src/hmr/content.rs | 2 +- .../turbopack-ecmascript/src/hmr/merger.rs | 7 ++-- .../turbopack-ecmascript/src/hmr/update.rs | 2 +- .../module_fragments/side_effects/module.rs | 2 +- .../src/references/amd.rs | 2 +- .../src/references/async_module.rs | 4 +- .../src/references/esm/export.rs | 2 +- .../src/references/hot_module.rs | 4 +- .../src/references/mod.rs | 34 +++++++---------- .../src/references/pattern_mapping.rs | 2 +- .../src/typescript/mod.rs | 2 +- .../turbopack-ecmascript/src/webpack/mod.rs | 2 +- .../crates/turbopack-node/src/evaluate.rs | 2 +- .../turbopack-node/src/process_pool/mod.rs | 6 +-- .../turbopack-node/src/transforms/postcss.rs | 2 +- .../turbopack-node/src/transforms/webpack.rs | 4 +- .../src/node_native_binding.rs | 12 +++--- .../turbopack-test-utils/src/snapshot.rs | 2 +- .../crates/turbopack/src/global_module_ids.rs | 2 +- 54 files changed, 129 insertions(+), 156 deletions(-) diff --git a/crates/next-api/src/aggregate_hmr.rs b/crates/next-api/src/aggregate_hmr.rs index c773cca54302..42833b9b3904 100644 --- a/crates/next-api/src/aggregate_hmr.rs +++ b/crates/next-api/src/aggregate_hmr.rs @@ -58,7 +58,7 @@ impl Version for AggregateHmrVersion { let version = TraitRef::cell(version.clone()); async move { let id = version.id().owned().await?; - Ok::<_, anyhow::Error>((path, id)) + anyhow::Ok((path, id)) } }) .try_join() @@ -93,7 +93,7 @@ impl AggregateHmrVersion { let content = *content; async move { let version = content.version().into_trait_ref().await?; - Ok::<_, anyhow::Error>((path, version)) + anyhow::Ok((path, version)) } }) .try_join() @@ -194,9 +194,9 @@ pub async fn diff_chunks_against( }; Some((path.clone(), *content, TraitRef::cell(prev))) }) - .map(|(path, content, prev)| async move { + .map(async |(path, content, prev)| { let update = content.update(prev).await?; - Ok::<_, anyhow::Error>((path, update)) + anyhow::Ok((path, update)) }) .try_join() .await?; diff --git a/crates/next-api/src/app.rs b/crates/next-api/src/app.rs index 6cc9e4a60862..dcc8d2289ecd 100644 --- a/crates/next-api/src/app.rs +++ b/crates/next-api/src/app.rs @@ -827,7 +827,7 @@ impl AppProject { .any(|route| route.as_str() == pathname.to_string()) }) }) - .map(|(pathname, app_entrypoint)| async { + .map(async |(pathname, app_entrypoint)| { Ok(( pathname.to_string().into(), app_entry_point_to_route(self, app_entrypoint.clone()) diff --git a/crates/next-api/src/client_references.rs b/crates/next-api/src/client_references.rs index 5f77173b3868..46a16a11e8d0 100644 --- a/crates/next-api/src/client_references.rs +++ b/crates/next-api/src/client_references.rs @@ -34,7 +34,7 @@ pub async fn map_client_references( let manifest = graph .await? .iter_reachable_modules()? - .map(|module| async move { + .map(async |module| { if let Some(client_reference_module) = ResolvedVc::try_downcast_type::(module) { diff --git a/crates/next-api/src/dynamic_imports.rs b/crates/next-api/src/dynamic_imports.rs index 3c0e4f92b852..c96284195464 100644 --- a/crates/next-api/src/dynamic_imports.rs +++ b/crates/next-api/src/dynamic_imports.rs @@ -54,7 +54,7 @@ pub(crate) async fn collect_next_dynamic_chunks( let chunking_availability = &chunking_availability; let dynamic_import_chunks = dynamic_import_entries .iter() - .map(|(dynamic_entry, parent_client_reference)| async move { + .map(async |(dynamic_entry, parent_client_reference)| { let module = ResolvedVc::upcast::>(*dynamic_entry); // This is the availability info for the parent chunk group, i.e. the client reference @@ -124,7 +124,7 @@ pub async fn map_next_dynamic( graph .await? .iter_reachable_modules()? - .map(|module| async move { + .map(async |module| { if let Some(dynamic_entry_module) = ResolvedVc::try_downcast_type::(module) && module.ident().await?.layer.as_ref().is_some_and(|layer| { diff --git a/crates/next-api/src/module_graph.rs b/crates/next-api/src/module_graph.rs index 67b924753d42..58568a60afff 100644 --- a/crates/next-api/src/module_graph.rs +++ b/crates/next-api/src/module_graph.rs @@ -103,7 +103,7 @@ impl NextDynamicGraphs { let result = self .0 .iter() - .map(|graph| async move { + .map(async |graph| { Ok(graph .get_next_dynamic_imports_for_endpoint(entry) .await? @@ -300,7 +300,7 @@ impl ServerActionsGraphs { let result = self .0 .iter() - .map(|graph| async move { + .map(async |graph| { graph .get_server_actions_for_endpoint(entry, rsc_asset_context) .owned() @@ -372,7 +372,7 @@ impl ServerActionsGraph { let actions = data .iter() - .map(|(module, (layer, actions))| async move { + .map(async |(module, (layer, actions))| { let actions = actions.await?; actions .actions diff --git a/crates/next-api/src/versioned_content_map.rs b/crates/next-api/src/versioned_content_map.rs index 105262d02a74..b6edcb316b39 100644 --- a/crates/next-api/src/versioned_content_map.rs +++ b/crates/next-api/src/versioned_content_map.rs @@ -124,7 +124,7 @@ impl VersionedContentMap { let rel = root.get_path_to(&path)?; Some((RcStr::from(rel), path)) }) - .map(|(name, path)| async move { + .map(async |(name, path)| { // Skip Redirect assets: they're symlinks with no file content, // so versioning them would bail with "not a file". let Some(asset) = *self.get_asset(path).await? else { @@ -337,7 +337,7 @@ async fn get_entries(assets: OperationVc) -> Result>> = client_references_chunks_ref .layout_segment_client_chunks .values() - .map(|&assets| async move { + .map(async |&assets| { let primary = assets.primary_assets().await?; Ok(primary.iter().copied().collect::>()) }) diff --git a/crates/next-core/src/pages_structure.rs b/crates/next-core/src/pages_structure.rs index 718a3223ab09..25704733d80f 100644 --- a/crates/next-core/src/pages_structure.rs +++ b/crates/next-core/src/pages_structure.rs @@ -241,12 +241,12 @@ async fn get_pages_structure_for_root_directory( next_router_path: next_router_path.clone(), items: items .into_iter() - .map(|(_, v)| async move { v.to_resolved().await }) + .map(|(_, v)| v.to_resolved()) .try_join() .await?, children: children .into_iter() - .map(|(_, v)| async move { v.to_resolved().await }) + .map(|(_, v)| v.to_resolved()) .try_join() .await?, } @@ -386,13 +386,13 @@ async fn get_pages_structure_for_directory( items: items .into_iter() .map(|(_, v)| v) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?, children: children .into_iter() .map(|(_, v)| v) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?, } diff --git a/crates/next-core/src/segment_config.rs b/crates/next-core/src/segment_config.rs index 5a7e85746704..15724d0e0f1f 100644 --- a/crates/next-core/src/segment_config.rs +++ b/crates/next-core/src/segment_config.rs @@ -1361,9 +1361,7 @@ async fn parse_segment_config_from_loader_tree_internal( let parallel_configs = loader_tree .parallel_routes .values() - .map(|loader_tree| async move { - Box::pin(parse_segment_config_from_loader_tree_internal(loader_tree)).await - }) + .map(|loader_tree| Box::pin(parse_segment_config_from_loader_tree_internal(loader_tree))) .try_join() .await?; diff --git a/crates/next-napi-bindings/src/next_api/project.rs b/crates/next-napi-bindings/src/next_api/project.rs index 3b284bdd3b0c..fdefb38e1a2d 100644 --- a/crates/next-napi-bindings/src/next_api/project.rs +++ b/crates/next-napi-bindings/src/next_api/project.rs @@ -1684,7 +1684,7 @@ async fn output_assets_operation( let endpoint_assets = endpoints .iter() - .map(|endpoint| async move { endpoint.output().await?.output_assets.await }) + .map(async |endpoint| endpoint.output().await?.output_assets.await) .try_join() .await?; diff --git a/turbopack/crates/turbopack-analyze/tests/split_chunk.rs b/turbopack/crates/turbopack-analyze/tests/split_chunk.rs index 8f9c9590fd65..1e20544f3a34 100644 --- a/turbopack/crates/turbopack-analyze/tests/split_chunk.rs +++ b/turbopack/crates/turbopack-analyze/tests/split_chunk.rs @@ -22,7 +22,7 @@ static REGISTRATION: Registration = register!(); #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn split_chunk() { - run_once(®ISTRATION, || async { + run_once(®ISTRATION, async || { let mut code = CodeBuilder::new(true, false); code += "Hello world!\n"; code += "This is a test file.\n"; diff --git a/turbopack/crates/turbopack-bench/src/lib.rs b/turbopack/crates/turbopack-bench/src/lib.rs index 8f27940201ab..fe71017e12cb 100644 --- a/turbopack/crates/turbopack-bench/src/lib.rs +++ b/turbopack/crates/turbopack-bench/src/lib.rs @@ -88,7 +88,7 @@ fn bench_startup_internal( |b, &(bundler, test_app)| { let test_app = &**test_app; let browser = &*browser; - b.to_async(&runtime).try_iter_custom(|iters, m| async move { + b.to_async(&runtime).try_iter_custom(async |iters, m| { let mut value = m.zero(); for _ in 0..iters { @@ -183,7 +183,7 @@ fn bench_hmr_internal( b.to_async(&runtime).try_iter_async( &runtime, - || async { + async || { let mut app = PreparedApp::new_without_copy( bundler, test_app.path().to_path_buf(), @@ -318,7 +318,7 @@ fn bench_hmr_internal( Ok((guard, value)) } }, - |guard| async move { + async |guard| { let hmr_is_happening = guard .page() .evaluate_expression("globalThis.HMR_IS_HAPPENING") @@ -484,7 +484,7 @@ fn bench_startup_cached_internal( |b, &(bundler, test_app)| { let test_app = &**test_app; let browser = &*browser; - b.to_async(&runtime).try_iter_custom(|iters, m| async move { + b.to_async(&runtime).try_iter_custom(async |iters, m| { // Run a complete build, shut down, and test running it again let mut app = PreparedApp::new(bundler, test_app.path().to_path_buf()).await?; diff --git a/turbopack/crates/turbopack-bench/src/util/mod.rs b/turbopack/crates/turbopack-bench/src/util/mod.rs index 460f092f8860..c6c9c4b33640 100644 --- a/turbopack/crates/turbopack-bench/src/util/mod.rs +++ b/turbopack/crates/turbopack-bench/src/util/mod.rs @@ -164,7 +164,7 @@ impl AsyncBencherExtension for AsyncBencher<'_, '_, A> { let log_progress = read_env_bool("TURBOPACK_BENCH_PROGRESS"); let routine = &routine; - self.iter_custom(|iters| async move { + self.iter_custom(async |iters| { let measurement = WallTime; let value = routine(iters, measurement).await.expect("routine failed"); if log_progress { @@ -208,7 +208,7 @@ impl AsyncBencherExtension for AsyncBencher<'_, '_, A> { input })))); - self.iter_custom(|iters| async move { + self.iter_custom(async |iters| { let measurement = WallTime; let input = input_mutex diff --git a/turbopack/crates/turbopack-cli/src/build/mod.rs b/turbopack/crates/turbopack-cli/src/build/mod.rs index c70afe7414d9..22723b4a6517 100644 --- a/turbopack/crates/turbopack-cli/src/build/mod.rs +++ b/turbopack/crates/turbopack-cli/src/build/mod.rs @@ -262,27 +262,20 @@ async fn build_internal( source_maps_type, ); - let entry_requests = (*entry_requests - .into_iter() - .map(|r| async move { - Ok(match r { - EntryRequest::Relative(p) => Request::relative( - p.clone().into(), - Default::default(), - Default::default(), - false, - ), - EntryRequest::Module(m, p) => Request::module( - m.clone().into(), - p.clone().into(), - Default::default(), - Default::default(), - ), - }) - }) - .try_join() - .await?) - .to_vec(); + let entry_requests = entry_requests.into_iter().map(|r| match r { + EntryRequest::Relative(p) => Request::relative( + p.clone().into(), + Default::default(), + Default::default(), + false, + ), + EntryRequest::Module(m, p) => Request::module( + m.clone().into(), + p.clone().into(), + Default::default(), + Default::default(), + ), + }); let origin = PlainResolveOrigin::new(asset_context, project_fs.root().await?.join("_")?).await?; @@ -292,7 +285,6 @@ async fn build_internal( let project_dir = &project_dir; let entries = async move { entry_requests - .into_iter() .map(|request_vc| { let origin_path = origin_path.clone(); async move { @@ -524,7 +516,7 @@ async fn build_internal( all_assets .iter() - .map(|c| async move { c.content().write(c.path().owned().await?).await }) + .map(async |c| c.content().write(c.path().owned().await?).await) .try_join() .await?; diff --git a/turbopack/crates/turbopack-cli/src/dev/web_entry_source.rs b/turbopack/crates/turbopack-cli/src/dev/web_entry_source.rs index 037c4b7976f3..23d80e995fd1 100644 --- a/turbopack/crates/turbopack-cli/src/dev/web_entry_source.rs +++ b/turbopack/crates/turbopack-cli/src/dev/web_entry_source.rs @@ -183,7 +183,7 @@ pub async fn create_web_entry_source( let entries: Vec<_> = entries .into_iter() - .map(|module| async move { + .map(async |module| { if let (Some(chunkable_module), Some(entry)) = ( ResolvedVc::try_sidecast::>(module), ResolvedVc::try_sidecast::>(module), diff --git a/turbopack/crates/turbopack-core/src/issue/mod.rs b/turbopack/crates/turbopack-core/src/issue/mod.rs index 2fa8628abc06..5ed42693725d 100644 --- a/turbopack/crates/turbopack-core/src/issue/mod.rs +++ b/turbopack/crates/turbopack-core/src/issue/mod.rs @@ -768,7 +768,7 @@ pub type PlainTrace = Vec; async fn into_plain_trace(traces: Vec>>) -> Result> { let mut plain_traces = traces .into_iter() - .map(|trace| async move { + .map(async |trace| { let mut plain_trace = trace .into_iter() .filter(|asset| { diff --git a/turbopack/crates/turbopack-core/src/module_graph/mod.rs b/turbopack/crates/turbopack-core/src/module_graph/mod.rs index 4954a9dda8e1..6d8480a32125 100644 --- a/turbopack/crates/turbopack-core/src/module_graph/mod.rs +++ b/turbopack/crates/turbopack-core/src/module_graph/mod.rs @@ -683,7 +683,7 @@ impl ModuleGraphImportTracer { .await? .modules .iter() - .map(|(&module, _)| async move { Ok((module.ident().await?.path.clone(), module)) }) + .map(async |(&module, _)| Ok((module.ident().await?.path.clone(), module))) .try_join() .await?; let mut map: FxHashMap>>> = @@ -711,7 +711,7 @@ impl ImportTracer for ModuleGraphImportTracer { return Ok(ImportTraces::cell(ImportTraces( modules .iter() - .map(|m| async move { + .map(async |m| { let Some(&module_idx) = graph.modules.get(m) else { // The only way this could really happen is if `path_to_modules` is computed // from a different graph than graph`. Just error out. @@ -2344,7 +2344,7 @@ pub mod tests { // test traversing backwards from 'd' which is only in the child graph let d_module = child_graph .enumerate_nodes() - .map(|(_index, module)| async move { + .map(async |(_index, module)| { Ok(match module { crate::module_graph::SingleModuleGraphNode::Module(module) => { if module.ident().to_string().owned().await? == "[test]/d.js" { @@ -2854,7 +2854,7 @@ pub mod tests { .await? .modules .keys() - .map(|m| async move { Ok((*m, m.ident().await?.path.path.clone())) }) + .map(async |m| Ok((*m, m.ident().await?.path.path.clone()))) .try_join() .await? .into_iter() diff --git a/turbopack/crates/turbopack-core/src/reference/mod.rs b/turbopack/crates/turbopack-core/src/reference/mod.rs index 93bd7254ff89..08ae40d7c893 100644 --- a/turbopack/crates/turbopack-core/src/reference/mod.rs +++ b/turbopack/crates/turbopack-core/src/reference/mod.rs @@ -131,7 +131,7 @@ pub async fn referenced_modules_and_affecting_sources( .references() .await? .iter() - .map(|reference| async { + .map(async |reference| { let trait_ref = reference.into_trait_ref().await?; let resolve_result = reference.resolve_reference().await?; if let Some(chunking_type) = &trait_ref.chunking_type() { @@ -141,7 +141,7 @@ pub async fn referenced_modules_and_affecting_sources( modules.extend( resolve_result .affecting_sources_iter() - .map(|source| async move { + .map(async |source| { Ok(ResolvedVc::upcast( RawModule::new(*source).to_resolved().await?, )) @@ -211,7 +211,7 @@ pub async fn primary_referenced_modules(module: Vc>) -> Result Result> { Ok(self - .map_module(|asset| async move { + .map_module(async |asset| { Ok(ModuleResolveResultItem::Module(ResolvedVc::upcast( RawModule::new(*asset).to_resolved().await?, ))) @@ -1138,7 +1138,7 @@ async fn realpath( result .symlinks .iter() - .map(|path| async move { + .map(async |path| { Ok(ResolvedVc::upcast( FileSource::new(path.clone()).to_resolved().await?, )) @@ -1540,7 +1540,7 @@ pub async fn resolve_raw( ) -> Result>> { Ok(matches .iter() - .map(|m| async move { + .map(async |m| { Ok(if let PatternMatch::File(request, path) = m { Some(to_result(request.clone(), path, collect_affecting_sources).await?) } else { @@ -1930,9 +1930,7 @@ async fn resolve_internal_inline( Request::Alternatives { requests } => { let results = requests .iter() - .map(|req| async { - resolve_internal_inline(lookup_path.clone(), **req, options).await - }) + .map(|req| resolve_internal_inline(lookup_path.clone(), **req, options)) .try_join() .await?; @@ -3169,7 +3167,7 @@ async fn resolved( result .symlinks .iter() - .map(|symlink| async move { + .map(async |symlink| { anyhow::Ok(ResolvedVc::upcast( FileSource::new(symlink.clone()).to_resolved().await?, )) diff --git a/turbopack/crates/turbopack-core/src/resolve/parse.rs b/turbopack/crates/turbopack-core/src/resolve/parse.rs index 639928917bc7..4f7ea158738f 100644 --- a/turbopack/crates/turbopack-core/src/resolve/parse.rs +++ b/turbopack/crates/turbopack-core/src/resolve/parse.rs @@ -440,7 +440,7 @@ impl Request { .copied() .map(|v| *v) .map(Request::as_relative) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?; Request::Alternatives { requests }.cell() @@ -500,7 +500,7 @@ impl Request { .iter() .copied() .map(|req| req.with_query(query.clone())) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?; Request::Alternatives { requests }.cell() @@ -578,7 +578,7 @@ impl Request { .iter() .copied() .map(|req| req.with_fragment(fragment.clone())) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?; Request::Alternatives { requests }.cell() @@ -690,7 +690,7 @@ impl Request { Request::Alternatives { requests } => { let requests = requests .iter() - .map(|req| async { req.append_path(suffix.clone()).to_resolved().await }) + .map(|req| req.append_path(suffix.clone()).to_resolved()) .try_join() .await?; Request::Alternatives { requests }.cell() diff --git a/turbopack/crates/turbopack-css/src/chunk/mod.rs b/turbopack/crates/turbopack-css/src/chunk/mod.rs index 39d0a75b4057..b198347cc7d4 100644 --- a/turbopack/crates/turbopack-css/src/chunk/mod.rs +++ b/turbopack/crates/turbopack-css/src/chunk/mod.rs @@ -190,7 +190,7 @@ impl CssChunk { } let assets = chunk_items .iter() - .map(|chunk_item| async move { + .map(async |chunk_item| { Ok(( rcstr!("chunk item"), chunk_item.content_ident().to_resolved().await?, @@ -257,7 +257,7 @@ impl OutputAssetsReference for CssChunk { let references = content .chunk_items .iter() - .map(|item| async { + .map(async |item| { let refs = item.references().await?; let single_css_chunk = if should_generate_single_item_chunks { Some(ResolvedVc::upcast( @@ -353,7 +353,7 @@ impl OutputChunk for CssChunk { .await?; let imports_chunk_items: Vec<_> = entries_chunk_items .iter() - .map(|&css_item| async move { + .map(async |&css_item| { Ok(css_item .content() .await? @@ -498,7 +498,7 @@ impl Introspectable for CssChunk { .await? .chunk_items .iter() - .map(|chunk_item| async move { + .map(async |chunk_item| { Ok(( rcstr!("entry module"), IntrospectableModule::new(chunk_item.module()) diff --git a/turbopack/crates/turbopack-dev-server/src/html.rs b/turbopack/crates/turbopack-dev-server/src/html.rs index eff1885b457a..f774a4f6014f 100644 --- a/turbopack/crates/turbopack-dev-server/src/html.rs +++ b/turbopack/crates/turbopack-dev-server/src/html.rs @@ -131,7 +131,7 @@ impl DevHtmlAsset { let all_chunk_groups = self .entries .iter() - .map(|entry| async move { + .map(async |entry| { let &DevHtmlEntry { chunkable_module, chunking_context, diff --git a/turbopack/crates/turbopack-dev-server/src/introspect/mod.rs b/turbopack/crates/turbopack-dev-server/src/introspect/mod.rs index 0201241f2d78..3fe6e017447b 100644 --- a/turbopack/crates/turbopack-dev-server/src/introspect/mod.rs +++ b/turbopack/crates/turbopack-dev-server/src/introspect/mod.rs @@ -135,7 +135,7 @@ impl GetContentSourceContent for IntrospectionSource { let has_children = !children.is_empty(); let children = children .iter() - .map(|(name, child)| async move { + .map(async |(name, child)| { let ty = child.ty().await; let ty = str_or_err(&ty); let title = child.title().await; diff --git a/turbopack/crates/turbopack-dev-server/src/source/asset_graph.rs b/turbopack/crates/turbopack-dev-server/src/source/asset_graph.rs index d30dbcabf9c9..79a53920472d 100644 --- a/turbopack/crates/turbopack-dev-server/src/source/asset_graph.rs +++ b/turbopack/crates/turbopack-dev-server/src/source/asset_graph.rs @@ -89,7 +89,7 @@ async fn expand( let mut assets_set = FxHashSet::default(); let root_assets_with_path = root_assets .iter() - .map(|&asset| async move { + .map(async |&asset| { let path = asset.path().await?; Ok((path, asset)) }) @@ -220,7 +220,7 @@ impl ContentSource for AssetGraphContentSource { )), ) }) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?; Ok(Vc::::cell(routes).merge()) @@ -307,7 +307,7 @@ impl Introspectable for AssetGraphContentSource { let root_assets = this.root_assets.await?; let root_asset_children = root_assets .iter() - .map(|&asset| async move { + .map(async |&asset| { Ok(( rcstr!("root"), IntrospectableOutputAsset::new(*asset).to_resolved().await?, @@ -320,7 +320,7 @@ impl Introspectable for AssetGraphContentSource { let expanded_asset_children = expanded_assets .values() .filter(|&a| !root_assets.contains(a)) - .map(|&asset| async move { + .map(async |&asset| { Ok(( rcstr!("inner"), IntrospectableOutputAsset::new(*asset).to_resolved().await?, @@ -364,7 +364,7 @@ impl Introspectable for FullyExpanded { let expanded_assets = expand(&*source.root_assets.await?, &source.root_path, None).await?; let children = expanded_assets .iter() - .map(|(_k, &v)| async move { + .map(async |(_k, &v)| { Ok(( rcstr!("asset"), IntrospectableOutputAsset::new(*v).to_resolved().await?, diff --git a/turbopack/crates/turbopack-dev-server/src/source/combined.rs b/turbopack/crates/turbopack-dev-server/src/source/combined.rs index 6cb1e5e6d308..25422f7bf207 100644 --- a/turbopack/crates/turbopack-dev-server/src/source/combined.rs +++ b/turbopack/crates/turbopack-dev-server/src/source/combined.rs @@ -27,7 +27,7 @@ impl ContentSource for CombinedContentSource { let all_routes = self .sources .iter() - .map(|s| async move { s.get_routes().to_resolved().await }) + .map(|s| s.get_routes().to_resolved()) .try_join() .await?; Ok(Vc::::cell(all_routes).merge()) @@ -51,7 +51,7 @@ impl Introspectable for CombinedContentSource { let titles = self .sources .iter() - .map(|&source| async move { + .map(async |&source| { Ok( if let Some(source) = ResolvedVc::try_sidecast::>(source) @@ -85,11 +85,7 @@ impl Introspectable for CombinedContentSource { self.sources .iter() .copied() - .map(|s| async move { Ok(ResolvedVc::try_sidecast::>(s)) }) - .try_join() - .await? - .into_iter() - .flatten() + .flat_map(ResolvedVc::try_sidecast::>) .map(|i| (rcstr!("source"), i)) .collect(), )) diff --git a/turbopack/crates/turbopack-dev-server/src/source/route_tree.rs b/turbopack/crates/turbopack-dev-server/src/source/route_tree.rs index 548ae62af5be..41966a4dc661 100644 --- a/turbopack/crates/turbopack-dev-server/src/source/route_tree.rs +++ b/turbopack/crates/turbopack-dev-server/src/source/route_tree.rs @@ -168,7 +168,7 @@ impl RouteTree { self.static_segments.extend( static_segments .into_iter() - .map(|(key, value)| async { + .map(async |(key, value)| { Ok(( key, if value.len() == 1 { diff --git a/turbopack/crates/turbopack-dev-server/src/source/router.rs b/turbopack/crates/turbopack-dev-server/src/source/router.rs index c87fb94b9e4b..a7a2c8c012e8 100644 --- a/turbopack/crates/turbopack-dev-server/src/source/router.rs +++ b/turbopack/crates/turbopack-dev-server/src/source/router.rs @@ -105,7 +105,7 @@ impl ContentSource for PrefixedRouterContentSource { Ok(Vc::::cell( inner_trees .chain(once(self.fallback.get_routes())) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?, ) diff --git a/turbopack/crates/turbopack-dev-server/src/source/static_assets.rs b/turbopack/crates/turbopack-dev-server/src/source/static_assets.rs index f596f67447d6..81aaedae44bd 100644 --- a/turbopack/crates/turbopack-dev-server/src/source/static_assets.rs +++ b/turbopack/crates/turbopack-dev-server/src/source/static_assets.rs @@ -66,7 +66,7 @@ async fn get_routes_from_directory(dir: FileSystemPath) -> Result> ), _ => None, }) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?; Ok(Vc::::cell(routes).merge()) diff --git a/turbopack/crates/turbopack-dev-server/src/source/wrapping_source.rs b/turbopack/crates/turbopack-dev-server/src/source/wrapping_source.rs index 55b681fbd94c..299422fdcb2e 100644 --- a/turbopack/crates/turbopack-dev-server/src/source/wrapping_source.rs +++ b/turbopack/crates/turbopack-dev-server/src/source/wrapping_source.rs @@ -54,7 +54,7 @@ async fn wrap_sources_operation( **s, *processor, )) }) - .map(|v| async move { v.to_resolved().await }) + .map(|v| v.to_resolved()) .try_join() .await?, )) diff --git a/turbopack/crates/turbopack-ecmascript/benches/references.rs b/turbopack/crates/turbopack-ecmascript/benches/references.rs index da5442df291c..87711e3cc5c4 100644 --- a/turbopack/crates/turbopack-ecmascript/benches/references.rs +++ b/turbopack/crates/turbopack-ecmascript/benches/references.rs @@ -145,7 +145,7 @@ fn bench_full(b: &mut Bencher, input: &BenchInput) { }); (tt, module) }, - |(tt, module)| async move { + async |(tt, module)| { tt.run_once(async move { // `analyze_ecmascript_module` performs eventually-consistent Vc reads. Reading // not-yet-settled state at the top level is fine for a throughput benchmark, but diff --git a/turbopack/crates/turbopack-ecmascript/src/chunk/code_module_ids_and_paths.rs b/turbopack/crates/turbopack-ecmascript/src/chunk/code_module_ids_and_paths.rs index d5cc9b06b89d..7dececa5fa93 100644 --- a/turbopack/crates/turbopack-ecmascript/src/chunk/code_module_ids_and_paths.rs +++ b/turbopack/crates/turbopack-ecmascript/src/chunk/code_module_ids_and_paths.rs @@ -62,7 +62,7 @@ pub async fn item_code_module_ids_and_paths( .await? .chunk_items .iter() - .map(|item| async { + .map(async |item| { Ok(( item.chunk_item.id().await?, item.chunk_item diff --git a/turbopack/crates/turbopack-ecmascript/src/chunk/content_entry.rs b/turbopack/crates/turbopack-ecmascript/src/chunk/content_entry.rs index 54dbdddf7363..3abe23600eef 100644 --- a/turbopack/crates/turbopack-ecmascript/src/chunk/content_entry.rs +++ b/turbopack/crates/turbopack-ecmascript/src/chunk/content_entry.rs @@ -71,7 +71,7 @@ impl EcmascriptChunkContentEntries { batch .chunk_items .iter() - .map(|item| async move { + .map(async |item| { Ok(( item.chunk_item.id().await?, EcmascriptChunkContentEntry::new( diff --git a/turbopack/crates/turbopack-ecmascript/src/chunk/mod.rs b/turbopack/crates/turbopack-ecmascript/src/chunk/mod.rs index cfa99b0a5669..c229066fda26 100644 --- a/turbopack/crates/turbopack-ecmascript/src/chunk/mod.rs +++ b/turbopack/crates/turbopack-ecmascript/src/chunk/mod.rs @@ -150,7 +150,7 @@ impl Chunk for EcmascriptChunk { let assets = chunk_items .iter() - .map(|&chunk_item| async move { + .map(async |&chunk_item| { Ok(( rcstr!("chunk item"), chunk_item.content_ident().to_resolved().await?, diff --git a/turbopack/crates/turbopack-ecmascript/src/chunk/placeable.rs b/turbopack/crates/turbopack-ecmascript/src/chunk/placeable.rs index c87c85565dd2..f1c0e1302e45 100644 --- a/turbopack/crates/turbopack-ecmascript/src/chunk/placeable.rs +++ b/turbopack/crates/turbopack-ecmascript/src/chunk/placeable.rs @@ -124,7 +124,7 @@ async fn side_effects_from_package_json( }) } }) - .map(|glob| async move { + .map(async |glob| { Ok(match glob { Either::Left(glob) => { match glob.to_resolved().await { diff --git a/turbopack/crates/turbopack-ecmascript/src/chunk_list/version.rs b/turbopack/crates/turbopack-ecmascript/src/chunk_list/version.rs index ffa4b8ba9085..87104fd1c777 100644 --- a/turbopack/crates/turbopack-ecmascript/src/chunk_list/version.rs +++ b/turbopack/crates/turbopack-ecmascript/src/chunk_list/version.rs @@ -38,7 +38,7 @@ impl Version for ChunkListVersion { .by_path .iter() .map(|(path, version)| (path, TraitRef::cell(version.clone()))) - .map(|(path, version)| async move { + .map(async |(path, version)| { let id = version.id().owned().await?; Ok((path, id)) }) diff --git a/turbopack/crates/turbopack-ecmascript/src/hmr/content.rs b/turbopack/crates/turbopack-ecmascript/src/hmr/content.rs index 9fb710a9ed38..669fe5d2d84f 100644 --- a/turbopack/crates/turbopack-ecmascript/src/hmr/content.rs +++ b/turbopack/crates/turbopack-ecmascript/src/hmr/content.rs @@ -31,7 +31,7 @@ impl EcmascriptMergedChunkContent { versions: self .contents .iter() - .map(|content| async move { content.ecmascript_chunk_version().await }) + .map(|content| content.ecmascript_chunk_version()) .try_join() .await?, } diff --git a/turbopack/crates/turbopack-ecmascript/src/hmr/merger.rs b/turbopack/crates/turbopack-ecmascript/src/hmr/merger.rs index f565a6596ad4..3f04c4e29138 100644 --- a/turbopack/crates/turbopack-ecmascript/src/hmr/merger.rs +++ b/turbopack/crates/turbopack-ecmascript/src/hmr/merger.rs @@ -1,5 +1,5 @@ use anyhow::{Result, bail}; -use turbo_tasks::{ResolvedVc, TryJoinIterExt, Vc}; +use turbo_tasks::{ResolvedVc, Vc}; use turbopack_core::version::{VersionedContent, VersionedContentMerger, VersionedContents}; use crate::hmr::{EcmascriptHmrChunkContent, content::EcmascriptMergedChunkContent}; @@ -28,7 +28,7 @@ impl VersionedContentMerger for EcmascriptChunkContentMerger { let contents = contents .await? .iter() - .map(|content| async move { + .map(|content| { if let Some(content) = ResolvedVc::try_sidecast::>(*content) { @@ -37,8 +37,7 @@ impl VersionedContentMerger for EcmascriptChunkContentMerger { bail!("expected Vc>") } }) - .try_join() - .await?; + .collect::>>()?; Ok(Vc::upcast(EcmascriptMergedChunkContent { contents }.cell())) } diff --git a/turbopack/crates/turbopack-ecmascript/src/hmr/update.rs b/turbopack/crates/turbopack-ecmascript/src/hmr/update.rs index 5af6997c42e6..e6a81642d826 100644 --- a/turbopack/crates/turbopack-ecmascript/src/hmr/update.rs +++ b/turbopack/crates/turbopack-ecmascript/src/hmr/update.rs @@ -237,7 +237,7 @@ pub async fn update_ecmascript_merged_chunk( let to_contents = content .contents .iter() - .map(|content| async move { + .map(async |content| { let entries = content.entries().await?; let version = content.ecmascript_chunk_version().await?; Ok((*content, entries, version)) diff --git a/turbopack/crates/turbopack-ecmascript/src/module_fragments/side_effects/module.rs b/turbopack/crates/turbopack-ecmascript/src/module_fragments/side_effects/module.rs index 655a8b10601d..35a35cf43994 100644 --- a/turbopack/crates/turbopack-ecmascript/src/module_fragments/side_effects/module.rs +++ b/turbopack/crates/turbopack-ecmascript/src/module_fragments/side_effects/module.rs @@ -93,7 +93,7 @@ impl Module for SideEffectsModule { references.extend( self.side_effects .iter() - .map(|side_effect| async move { + .map(async |side_effect| { Ok(ResolvedVc::upcast( SingleChunkableModuleReference::new( *ResolvedVc::upcast(*side_effect), diff --git a/turbopack/crates/turbopack-ecmascript/src/references/amd.rs b/turbopack/crates/turbopack-ecmascript/src/references/amd.rs index 4b2c1e2174fe..9c357771b75d 100644 --- a/turbopack/crates/turbopack-ecmascript/src/references/amd.rs +++ b/turbopack/crates/turbopack-ecmascript/src/references/amd.rs @@ -158,7 +158,7 @@ impl AmdDefineWithDependenciesCodeGen { let resolved_elements = self .dependencies_requests .iter() - .map(|element| async move { + .map(async |element| { Ok(match element { AmdDefineDependencyElement::Request { request, diff --git a/turbopack/crates/turbopack-ecmascript/src/references/async_module.rs b/turbopack/crates/turbopack-ecmascript/src/references/async_module.rs index 9b8ceb3e4eb3..bb16e8088c7f 100644 --- a/turbopack/crates/turbopack-ecmascript/src/references/async_module.rs +++ b/turbopack/crates/turbopack-ecmascript/src/references/async_module.rs @@ -120,7 +120,7 @@ impl AsyncModule { let reference_idents = references .await? .iter() - .map(|r| async { + .map(async |r| { let Some(referenced_asset) = get_inherit_async_referenced_asset(*r).await? else { return Ok(None); }; @@ -173,7 +173,7 @@ impl AsyncModule { && references .await? .iter() - .map(|r| async { + .map(async |r| { let Some(referenced_asset) = get_inherit_async_referenced_asset(*r).await? else { return Ok(false); diff --git a/turbopack/crates/turbopack-ecmascript/src/references/esm/export.rs b/turbopack/crates/turbopack-ecmascript/src/references/esm/export.rs index 2f032e3a5b3c..3dc2cc3644d5 100644 --- a/turbopack/crates/turbopack-ecmascript/src/references/esm/export.rs +++ b/turbopack/crates/turbopack-ecmascript/src/references/esm/export.rs @@ -375,7 +375,7 @@ async fn get_all_export_names( let star_export_names = exports .star_exports .iter() - .map(|esm_ref| async { + .map(async |esm_ref| { Ok( if let ReferencedAsset::Some(m) = ReferencedAsset::from_resolve_result(esm_ref.resolve_reference()).await? diff --git a/turbopack/crates/turbopack-ecmascript/src/references/hot_module.rs b/turbopack/crates/turbopack-ecmascript/src/references/hot_module.rs index 9c23c1555f55..e850dd2839a1 100644 --- a/turbopack/crates/turbopack-ecmascript/src/references/hot_module.rs +++ b/turbopack/crates/turbopack-ecmascript/src/references/hot_module.rs @@ -135,7 +135,7 @@ impl ModuleHotReferenceCodeGen { let resolved_ids: Vec> = self .references .iter() - .map(|reference| async move { + .map(async |reference| { let r = reference.await?; let resolve_result = reference.resolve_reference(); PatternMapping::resolve_request( @@ -156,7 +156,7 @@ impl ModuleHotReferenceCodeGen { let esm_reimports: Vec> = self .esm_references .iter() - .map(|esm_ref| async move { + .map(async |esm_ref| { let Some(esm_ref) = esm_ref else { return Ok(None); }; diff --git a/turbopack/crates/turbopack-ecmascript/src/references/mod.rs b/turbopack/crates/turbopack-ecmascript/src/references/mod.rs index b4e896815001..d6a341a7c5d1 100644 --- a/turbopack/crates/turbopack-ecmascript/src/references/mod.rs +++ b/turbopack/crates/turbopack-ecmascript/src/references/mod.rs @@ -1640,17 +1640,14 @@ async fn handle_call<'a, G: Fn(BumpVec<'a, Effect<'a>>) + Send + Sync>( let linked_args_cache = OnceCell::new(); // Create the lazy linking closure that will be passed to handle_well_known_function_call - let linked_args = || async { - linked_args_cache - .get_or_try_init(|| async { - unlinked_args - .iter() - .map(|arg| arg.clone_in(state.arena.get_or_default())) - .map(|arg| state.link_value(arg, ImportAttributes::empty_ref())) - .try_join() - .await - }) - .await + let linked_args = || { + linked_args_cache.get_or_try_init(|| { + unlinked_args + .iter() + .map(|arg| arg.clone_in(state.arena.get_or_default())) + .map(|arg| state.link_value(arg, ImportAttributes::empty_ref())) + .try_join() + }) }; match func { @@ -4014,15 +4011,12 @@ async fn require_resolve_visitor<'a>( ) .to_resolved() .await?; - let mut values = - resolved - .await? - .primary_sources() - .map(|source| async move { - Ok(require_resolve(source.ident().await?.path.clone()).into()) - }) - .try_join() - .await?; + let mut values = resolved + .await? + .primary_sources() + .map(async |source| Ok(require_resolve(source.ident().await?.path.clone()).into())) + .try_join() + .await?; match values.len() { 0 => JsValue::unknown( diff --git a/turbopack/crates/turbopack-ecmascript/src/references/pattern_mapping.rs b/turbopack/crates/turbopack-ecmascript/src/references/pattern_mapping.rs index ff88a59f408e..3666cc0028e7 100644 --- a/turbopack/crates/turbopack-ecmascript/src/references/pattern_mapping.rs +++ b/turbopack/crates/turbopack-ecmascript/src/references/pattern_mapping.rs @@ -493,7 +493,7 @@ impl PatternMapping { .collect(); let map = items .into_iter() - .map(|(k, v)| async move { + .map(async |(k, v)| { let single_pattern_mapping = to_single_pattern_mapping( origin, chunking_context, diff --git a/turbopack/crates/turbopack-ecmascript/src/typescript/mod.rs b/turbopack/crates/turbopack-ecmascript/src/typescript/mod.rs index 0b1d036ac13d..016e8385a11b 100644 --- a/turbopack/crates/turbopack-ecmascript/src/typescript/mod.rs +++ b/turbopack/crates/turbopack-ecmascript/src/typescript/mod.rs @@ -62,7 +62,7 @@ impl Module for TsConfigModuleAsset { references.extend( configs[1..] .iter() - .map(|(_, config_asset)| async move { + .map(async |(_, config_asset)| { Ok(ResolvedVc::upcast( TsExtendsReference::new(**config_asset) .to_resolved() diff --git a/turbopack/crates/turbopack-ecmascript/src/webpack/mod.rs b/turbopack/crates/turbopack-ecmascript/src/webpack/mod.rs index 81ce1b87af06..2f3516c10881 100644 --- a/turbopack/crates/turbopack-ecmascript/src/webpack/mod.rs +++ b/turbopack/crates/turbopack-ecmascript/src/webpack/mod.rs @@ -207,7 +207,7 @@ impl ModuleReference for WebpackRuntimeAssetReference { Ok(resolved .await? - .map_module(|source| async move { + .map_module(async |source| { Ok(ModuleResolveResultItem::Module(ResolvedVc::upcast( WebpackModuleAsset::new( *source, diff --git a/turbopack/crates/turbopack-node/src/evaluate.rs b/turbopack/crates/turbopack-node/src/evaluate.rs index 1a216444677c..0a3f15592435 100644 --- a/turbopack/crates/turbopack-node/src/evaluate.rs +++ b/turbopack/crates/turbopack-node/src/evaluate.rs @@ -382,7 +382,7 @@ pub async fn custom_evaluate(evaluate_context: impl EvaluateContext) -> Result Result { let bytes = self - .with_process(|process| async move { - process.recv().await.context("failed to receive message") - }) + .with_process(async |process| process.recv().await.context("failed to receive message")) .await?; Ok(bytes) } async fn send(&mut self, message: Bytes) -> Result<()> { - self.with_process(|process| async move { + self.with_process(async |process| { timeout(Duration::from_secs(30), process.send(message)) .await .context("timeout while sending message")? diff --git a/turbopack/crates/turbopack-node/src/transforms/postcss.rs b/turbopack/crates/turbopack-node/src/transforms/postcss.rs index b31abb3b6583..223d0f2acd3e 100644 --- a/turbopack/crates/turbopack-node/src/transforms/postcss.rs +++ b/turbopack/crates/turbopack-node/src/transforms/postcss.rs @@ -233,7 +233,7 @@ async fn extra_configs_changed( let configs = config_paths .into_iter() - .map(|path| async move { + .map(async |path| { Ok( if matches!(&*path.get_type().await?, FileSystemEntryType::File) { match *asset_context diff --git a/turbopack/crates/turbopack-node/src/transforms/webpack.rs b/turbopack/crates/turbopack-node/src/transforms/webpack.rs index 83e2e5cb4f0d..8f33be5e8616 100644 --- a/turbopack/crates/turbopack-node/src/transforms/webpack.rs +++ b/turbopack/crates/turbopack-node/src/transforms/webpack.rs @@ -622,11 +622,11 @@ impl EvaluateContext for WebpackLoaderContext { .try_join(); let file_subscriptions = file_paths .iter() - .map(|p| async move { self.cwd.join(p)?.read().await }) + .map(async |p| self.cwd.join(p)?.read().await) .try_join(); let directory_subscriptions = directories .iter() - .map(|(dir, glob)| async move { + .map(async |(dir, glob)| { self.cwd .join(dir)? .track_glob(Glob::new(glob.clone(), GlobOptions::default()), false) diff --git a/turbopack/crates/turbopack-resolve/src/node_native_binding.rs b/turbopack/crates/turbopack-resolve/src/node_native_binding.rs index 18293fa4c80f..6f19fbe2d44f 100644 --- a/turbopack/crates/turbopack-resolve/src/node_native_binding.rs +++ b/turbopack/crates/turbopack-resolve/src/node_native_binding.rs @@ -204,7 +204,7 @@ async fn resolve_node_pre_gyp_files( return Ok(*ModuleResolveResult::modules_with_affecting_sources( sources .into_iter() - .map(|(key, source)| async move { + .map(async |(key, source)| { Ok(( RequestKey::new(key), ResolvedVc::upcast(RawModule::new(source).to_resolved().await?), @@ -214,9 +214,7 @@ async fn resolve_node_pre_gyp_files( .await?, affecting_paths .into_iter() - .map(|p| async move { - anyhow::Ok(ResolvedVc::upcast(FileSource::new(p).to_resolved().await?)) - }) + .map(async |p| Ok(ResolvedVc::upcast(FileSource::new(p).to_resolved().await?))) .try_join() .await?, )); @@ -323,7 +321,7 @@ async fn resolve_node_gyp_build_files( return Ok(*ModuleResolveResult::modules_with_affecting_sources( resolved .into_iter() - .map(|(key, source)| async move { + .map(async |(key, source)| { Ok(( RequestKey::new(key), ResolvedVc::upcast(RawModule::new(*source).to_resolved().await?), @@ -434,7 +432,7 @@ async fn resolve_node_bindings_files( root_context_dir = parent; } - let try_path = |sub_path: RcStr| async move { + let try_path = async |sub_path: RcStr| { let path = root_context_dir.join(&sub_path)?; Ok( if matches!(*path.get_type().await?, FileSystemEntryType::File) { @@ -454,7 +452,7 @@ async fn resolve_node_bindings_files( let modules = BINDINGS_TRY .iter() - .map(|try_dir| try_path.clone()(format!("{}/{}", try_dir, file_name).into())) + .map(|try_dir| try_path(format!("{}/{}", try_dir, file_name).into())) .try_flat_join() .await?; Ok(*ModuleResolveResult::modules(modules)) diff --git a/turbopack/crates/turbopack-test-utils/src/snapshot.rs b/turbopack/crates/turbopack-test-utils/src/snapshot.rs index 77926a6a23ac..06391d4a8d33 100644 --- a/turbopack/crates/turbopack-test-utils/src/snapshot.rs +++ b/turbopack/crates/turbopack-test-utils/src/snapshot.rs @@ -187,7 +187,7 @@ async fn diff_paths( ) -> Result> { let mut map = left .iter() - .map(|p| async move { Ok((p.path.clone(), p.clone())) }) + .map(async |p| Ok((p.path.clone(), p.clone()))) .try_join() .await? .iter() diff --git a/turbopack/crates/turbopack/src/global_module_ids.rs b/turbopack/crates/turbopack/src/global_module_ids.rs index 7590a40394de..f8a7baedadae 100644 --- a/turbopack/crates/turbopack/src/global_module_ids.rs +++ b/turbopack/crates/turbopack/src/global_module_ids.rs @@ -49,7 +49,7 @@ pub async fn get_global_module_id_strategy( .into_iter() .map(|m| m.ident()) .chain(async_idents.into_iter()) - .map(|ident| async move { + .map(async |ident| { let ident = ident.to_resolved().await?; let ident_str = ident.to_string().await?; let hash = hash_xxh3_hash64(&ident_str); From f5841365b81194a364dc86d686934b3ecf560418 Mon Sep 17 00:00:00 2001 From: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com> Date: Fri, 21 Aug 2026 20:26:10 +0200 Subject: [PATCH 6/7] Turbopack: trace graceful-fs calls (#97694) --- .../src/analyzer/well_known/kinds.rs | 6 ++++++ .../src/analyzer/well_known/mod.rs | 7 ++++++- turbopack/crates/turbopack-ecmascript/src/utils.rs | 1 + .../test/unit/asset-fs-extra/output.js | 2 +- .../test/unit/asset-graceful-fs/output.js | 11 ++++++----- turbopack/crates/turbopack-tracing/tests/unit.rs | 2 +- 6 files changed, 21 insertions(+), 8 deletions(-) diff --git a/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/kinds.rs b/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/kinds.rs index e2d262b64ce2..22c2fdbc3395 100644 --- a/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/kinds.rs +++ b/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/kinds.rs @@ -13,6 +13,8 @@ pub enum WellKnownObjectKind { FsModulePromises, FsExtraModule, FsExtraModuleDefault, + GracefulFsModule, + GracefulFsModuleDefault, ModuleModule, ModuleModuleDefault, UrlModule, @@ -84,6 +86,10 @@ impl WellKnownObjectKind { "fs-extra", "The Node.js fs-extra module: https://github.com/jprichardson/node-fs-extra", ), + Self::GracefulFsModule | Self::GracefulFsModuleDefault => ( + "graceful-fs", + "The Node.js graceful-fs module: https://github.com/isaacs/node-graceful-fs", + ), Self::FsModulePromises => ( "fs/promises", "The Node.js fs module: https://nodejs.org/api/fs.html#promises-api", diff --git a/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/mod.rs b/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/mod.rs index cd0459a6b473..1f5dfaa2f0b7 100644 --- a/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/mod.rs +++ b/turbopack/crates/turbopack-ecmascript/src/analyzer/well_known/mod.rs @@ -696,7 +696,9 @@ async fn well_known_object_member<'a>( } WellKnownObjectKind::FsModule | WellKnownObjectKind::FsModuleDefault - | WellKnownObjectKind::FsModulePromises => { + | WellKnownObjectKind::FsModulePromises + | WellKnownObjectKind::GracefulFsModule + | WellKnownObjectKind::GracefulFsModuleDefault => { fs_module_member(arena.get_or_default(), kind, prop) } WellKnownObjectKind::FsExtraModule | WellKnownObjectKind::FsExtraModuleDefault => { @@ -935,6 +937,9 @@ fn fs_extra_module_member<'a>( word.into(), )); } + (.., "readdir" | "readdirSync") => { + return JsValue::WellKnownFunction(WellKnownFunctionKind::FsReadDir); + } // fs-extra specific ( .., diff --git a/turbopack/crates/turbopack-ecmascript/src/utils.rs b/turbopack/crates/turbopack-ecmascript/src/utils.rs index 75335ec6dde9..98e42635b4e7 100644 --- a/turbopack/crates/turbopack-ecmascript/src/utils.rs +++ b/turbopack/crates/turbopack-ecmascript/src/utils.rs @@ -268,6 +268,7 @@ pub fn module_value_to_well_known_object<'a>(module_value: &ModuleValue) -> Opti b"resolve-from" => JsValue::WellKnownFunction(WellKnownFunctionKind::NodeResolveFrom), b"@grpc/proto-loader" => JsValue::WellKnownObject(WellKnownObjectKind::NodeProtobufLoader), b"fs-extra" => JsValue::WellKnownObject(WellKnownObjectKind::FsExtraModule), + b"graceful-fs" => JsValue::WellKnownObject(WellKnownObjectKind::GracefulFsModule), _ => return None, }) } diff --git a/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-fs-extra/output.js b/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-fs-extra/output.js index 40630b786d94..c54b7fc8d612 100644 --- a/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-fs-extra/output.js +++ b/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-fs-extra/output.js @@ -1,5 +1,5 @@ ;[ - // TODO + // TODO we currently don't npm install in the unit test fixture // 'node_modules/fs-extra/lib/copy/copy-sync.js', // 'node_modules/fs-extra/lib/copy/copy.js', // 'node_modules/fs-extra/lib/copy/index.js', diff --git a/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-graceful-fs/output.js b/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-graceful-fs/output.js index ed359a37b7cb..08b2d90f309a 100644 --- a/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-graceful-fs/output.js +++ b/turbopack/crates/turbopack-tracing/tests/node-file-trace/test/unit/asset-graceful-fs/output.js @@ -1,9 +1,10 @@ ;[ - 'node_modules/graceful-fs/clone.js', - 'node_modules/graceful-fs/graceful-fs.js', - 'node_modules/graceful-fs/legacy-streams.js', - 'node_modules/graceful-fs/package.json', - 'node_modules/graceful-fs/polyfills.js', + // TODO we currently don't npm install in the unit test fixture + // 'node_modules/graceful-fs/clone.js', + // 'node_modules/graceful-fs/graceful-fs.js', + // 'node_modules/graceful-fs/legacy-streams.js', + // 'node_modules/graceful-fs/package.json', + // 'node_modules/graceful-fs/polyfills.js', 'package.json', 'test/unit/asset-graceful-fs/asset.txt', 'test/unit/asset-graceful-fs/input.js', diff --git a/turbopack/crates/turbopack-tracing/tests/unit.rs b/turbopack/crates/turbopack-tracing/tests/unit.rs index 92f620b2aa2e..131a1266f5fa 100644 --- a/turbopack/crates/turbopack-tracing/tests/unit.rs +++ b/turbopack/crates/turbopack-tracing/tests/unit.rs @@ -67,7 +67,7 @@ static ALLOC: turbo_tasks_malloc::TurboMalloc = turbo_tasks_malloc::TurboMalloc; #[case::asset_fs_inlining("asset-fs-inlining")] #[case::asset_fs_inlining_multi("asset-fs-inlining-multi")] #[case::asset_fs_logical("asset-fs-logical")] -// #[case::asset_graceful_fs("asset-graceful-fs")] +#[case::asset_graceful_fs("asset-graceful-fs")] #[case::asset_node_require("asset-node-require")] #[case::asset_package_json("asset-package-json")] #[case::asset_symlink("asset-symlink")] From 43273a1d21a35646ec43230e05727e18ac258396 Mon Sep 17 00:00:00 2001 From: Janka Uryga Date: Fri, 21 Aug 2026 21:08:38 +0200 Subject: [PATCH 7/7] [PPF] Instant validation for unstable_navigation() (#97309) Adds discriminated error messages for `unstable_navigation()` used outside of Suspense. Previously we only had to discriminate two kinds of errors in each validation flow: - App Shell -- 1. link data (if it resolves in `Runtime`) or 2. dynamic data (if it resolves in `Dynamic`) - Static Shell -- 1. runtime data (if it resolves in `Runtime`) or 2. dynamic data (if it resolves in `Dynamic`) However, with `navigation()` (96908), the App Shell flow needs to distinguish three kinds: 1. link data, 2. navigation (if it resolves in `NavigationRuntime`), and 3. dynamic data. This complicates the validation code a bit, because we can no longer get away with one retry. To accommodate this, i restructured the validation code to no longer recurse as a method of retrying. We just loop over an array that defines what order we should try the stages in + what kind of hole shows up in each stage. I've also removed `hasAmbiguousErrors` and the associated logic. since we're no longer mixing Static+Runtime segments in a runtime prefetch, this is no longer relevant -- all segments used for validation use one stage, so whether or not the error is ambiguous only depends on the stage. For ease of reviewing, this is split into two commits: 1. updating the (many) places that need a new `DynamicHoleKind` in our error messages. This one has a lot of pretty mechanical changes, and was also machine-reviewed for consistency already, but the messages themselves are worth looking at (we'll also need to update docs -- leaving that for a follow-up) 2. the actual instant validation changes, i.e. using the new `DynamicHoleKind` and adding the new pass --- packages/next/errors.json | 6 +- .../instant/instant-guidance-data.test.ts | 124 ++++++----- .../instant/instant-guidance-data.ts | 180 ++++++++++++++-- .../components/instant/instant-guidance.tsx | 80 +++---- .../dev-overlay/container/errors.test.ts | 94 ++++++-- .../dev-overlay/container/errors.tsx | 173 ++++++++++----- .../next/src/server/app-render/app-render.tsx | 203 +++++++++++------- .../app-render/blocking-route-messages.ts | 36 ++++ .../app-render/dev-validation-scheduler.ts | 9 +- .../server/app-render/dynamic-rendering.ts | 73 +++++-- .../instant-validation/boundary-tracking.tsx | 7 +- .../instant-validation/instant-validation.tsx | 82 ++----- .../head-and-reporting.util.ts | 168 +++++++-------- 13 files changed, 790 insertions(+), 445 deletions(-) diff --git a/packages/next/errors.json b/packages/next/errors.json index 6ab0ba34ae0c..94e088a4ca29 100644 --- a/packages/next/errors.json +++ b/packages/next/errors.json @@ -1490,5 +1490,9 @@ "1489": "Route %s used \\`unstable_prefetch()\\` inside a function cached with \\`unstable_cache()\\`. The \\`unstable_prefetch()\\` function is used to indicate the subsequent code must not run in the app shell, but \\`unstable_cache()\\` caches must be able to be produced before a prefetch, so this function is not allowed in this scope. See more info here: https://nextjs.org/docs/app/api-reference/functions/unstable_cache", "1490": "Route %s used \\`unstable_prefetch()\\` inside \\`after()\\` while rendering. The \\`unstable_prefetch()\\` function is used to indicate the subsequent code must not run in the app shell, but \\`after()\\` executes after the request, so this function is not allowed in this scope. See more info here: https://nextjs.org/docs/app/api-reference/functions/after", "1491": "Route %s used \\`unstable_prefetch()\\`, which requires Cache Components to be enabled. Learn more: https://nextjs.org/docs/app/api-reference/config/next-config-js/cacheComponents", - "1492": "Route %s used \\`unstable_prefetch()\\` inside \"use cache: private\". This is not currently supported. Instead, move the \"use cache\" directive to a function that's called below \\`await unstable_prefetch()\\`, so that the cached content is deferred to the prefetch without caching the stage boundary itself. See more info here: https://nextjs.org/docs/messages/next-request-in-use-cache" + "1492": "Route %s used \\`unstable_prefetch()\\` inside \"use cache: private\". This is not currently supported. Instead, move the \"use cache\" directive to a function that's called below \\`await unstable_prefetch()\\`, so that the cached content is deferred to the prefetch without caching the stage boundary itself. See more info here: https://nextjs.org/docs/messages/next-request-in-use-cache", + "1493": "Route \"%s\": Next.js encountered \\`unstable_navigation()\\` in \\`generateViewport()\\`.\\n\\n\\`unstable_navigation()\\` in \\`generateViewport()\\` prevents creating a shell, leading to a slower user experience.\\n\\nWays to fix this:\\n - [static] Use a static viewport export instead of \\`generateViewport()\\`\\n - [block] Set \\`export const instant = false\\` to allow a blocking route\\n\\nLearn more: https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime", + "1494": "Route \"%s\": Next.js encountered \\`unstable_navigation()\\` in \\`generateMetadata()\\`.\\n\\nThis route's metadata is blocked, but the rest of its content can be prefetched. \\`unstable_navigation()\\` called in \\`generateMetadata()\\` prevents it from being prefetched.\\n\\nWays to fix this:\\n - [static] Use a static metadata export instead of \\`generateMetadata()\\`\\n - [dynamic] Render a marker component that calls \\`await connection()\\` inside \\`\\` on the page\\n\\nLearn more: https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime", + "1495": "Route \"%s\": Next.js encountered \\`unstable_navigation()\\` during prerendering or a navigation.\\n\\n\\`unstable_navigation()\\` called outside of \\`\\` may prevent the navigation from being instant, leading to a slower user experience.\\n\\nWays to fix this:\\n - [stream] Provide a placeholder with \\`\\` around the data access\\n - [block] Set \\`export const instant = false\\` to allow a blocking route\\n\\nLearn more: https://nextjs.org/docs/messages/instant-shell-url-data", + "1496": "%s segments do not unblock new data in %s prefetches" } diff --git a/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.test.ts b/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.test.ts index 3c20d77de6b9..02d38333d47a 100644 --- a/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.test.ts +++ b/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.test.ts @@ -2,14 +2,17 @@ import { createRuntimeBodyError, createDynamicBodyError, createRuntimeBodyErrorInNavigation, - createDynamicBodyErrorInNavigation, createLinkBodyErrorInNavigation, + createNavigationBodyErrorInNavigation, + createDynamicBodyErrorInNavigation, createRuntimeMetadataError, - createDynamicMetadataError, createLinkMetadataError, + createNavigationMetadataError, + createDynamicMetadataError, createRuntimeViewportError, - createDynamicViewportError, createLinkViewportError, + createNavigationViewportError, + createDynamicViewportError, } from '../../../../server/app-render/blocking-route-messages' import { createLinkPrefetchPartialError } from '../../../../shared/lib/instant-messages' import { @@ -22,6 +25,13 @@ import { type GuidanceVariant, } from './instant-guidance-data' +const GUIDANCE_VARIANTS = [ + 'runtime', + 'link', + 'navigation', + 'dynamic', +] as const satisfies GuidanceVariant[] + function tagsFromMessage(message: string): string[] { return Array.from(message.matchAll(/^\s*-\s*\[([a-z]+)\]/gm)).map((m) => m[1]) } @@ -47,6 +57,7 @@ describe('instant-guidance-data card ordering', () => { 'blocking-route', 'dynamic', ], + [ 'blocking-route runtime in navigation', createRuntimeBodyErrorInNavigation('/x').message, @@ -54,53 +65,74 @@ describe('instant-guidance-data card ordering', () => { 'runtime', ], [ - 'blocking-route dynamic in navigation', - createDynamicBodyErrorInNavigation('/x').message, + 'blocking-route link in navigation', + createLinkBodyErrorInNavigation('/x').message, 'blocking-route', - 'dynamic', + 'link', ], [ - 'blocking-route link', - createLinkBodyErrorInNavigation('/x').message, + 'blocking-route navigation in navigation', + createNavigationBodyErrorInNavigation('/x').message, 'blocking-route', - 'link', + 'navigation', ], + [ + 'blocking-route dynamic in navigation', + createDynamicBodyErrorInNavigation('/x').message, + 'blocking-route', + 'dynamic', + ], + [ 'metadata runtime', createRuntimeMetadataError('/x').message, 'metadata', 'runtime', ], - [ - 'metadata dynamic', - createDynamicMetadataError('/x').message, - 'metadata', - 'dynamic', - ], [ 'metadata link', createLinkMetadataError('/x').message, 'metadata', 'link', ], + [ + 'metadata navigation', + createNavigationMetadataError('/x').message, + 'metadata', + 'navigation', + ], + [ + 'metadata dynamic', + createDynamicMetadataError('/x').message, + 'metadata', + 'dynamic', + ], + [ 'viewport runtime', createRuntimeViewportError('/x').message, 'viewport', 'runtime', ], - [ - 'viewport dynamic', - createDynamicViewportError('/x').message, - 'viewport', - 'dynamic', - ], [ 'viewport link', createLinkViewportError('/x').message, 'viewport', 'link', ], + [ + 'viewport navigation', + createNavigationViewportError('/x').message, + 'viewport', + 'navigation', + ], + [ + 'viewport dynamic', + createDynamicViewportError('/x').message, + 'viewport', + 'dynamic', + ], + [ 'link-prefetch-partial', createLinkPrefetchPartialError('/x').message, @@ -117,17 +149,12 @@ describe('instant-guidance-data card ordering', () => { describe('instant-guidance-data card links', () => { it('every card.link ends with #card.id', () => { - const variants: Array<[GuidanceKind, GuidanceVariant]> = [ - ['blocking-route', 'runtime'], - ['blocking-route', 'dynamic'], - ['blocking-route', 'link'], + type Item = [GuidanceKind, GuidanceVariant] + const variants: Array = [ + ...GUIDANCE_VARIANTS.map((v) => ['blocking-route', v] as Item), + ...GUIDANCE_VARIANTS.map((v) => ['metadata', v] as Item), + ...GUIDANCE_VARIANTS.map((v) => ['viewport', v] as Item), ['client-hook', 'runtime'], - ['metadata', 'runtime'], - ['metadata', 'dynamic'], - ['metadata', 'link'], - ['viewport', 'runtime'], - ['viewport', 'dynamic'], - ['viewport', 'link'], ['unrendered-segment', 'runtime'], ['link-prefetch-partial', 'runtime'], ] @@ -143,20 +170,18 @@ describe('instant-guidance-data card links', () => { describe('instant-guidance-data card invariants', () => { function allCards() { const cards = [] - const variants: Array<[GuidanceKind, GuidanceVariant, string?]> = [ - ['blocking-route', 'runtime'], - ['blocking-route', 'dynamic'], + type Item = [GuidanceKind, GuidanceVariant, string?] + const variants: Array = [ + ...GUIDANCE_VARIANTS.map((v) => ['blocking-route', v] as Item), ['blocking-route', 'dynamic', 'connection'], - ['blocking-route', 'link'], - ['client-hook', 'runtime'], - ['metadata', 'runtime'], - ['metadata', 'dynamic'], + + ...GUIDANCE_VARIANTS.map((v) => ['metadata', v] as Item), ['metadata', 'dynamic', 'connection'], - ['metadata', 'link'], - ['viewport', 'runtime'], - ['viewport', 'dynamic'], + + ...GUIDANCE_VARIANTS.map((v) => ['viewport', v] as Item), ['viewport', 'dynamic', 'connection'], - ['viewport', 'link'], + + ['client-hook', 'runtime'], ['unrendered-segment', 'runtime'], ['link-prefetch-partial', 'runtime'], ] @@ -214,17 +239,12 @@ describe('instant-guidance-data dispatcher', () => { it('every group in FIX_CARD_GROUPS is used by at least one card', () => { const used = new Set() - const variants: Array<[GuidanceKind, GuidanceVariant]> = [ - ['blocking-route', 'runtime'], - ['blocking-route', 'dynamic'], - ['blocking-route', 'link'], + type Item = [GuidanceKind, GuidanceVariant] + const variants: Array = [ + ...GUIDANCE_VARIANTS.map((v) => ['blocking-route', v] as Item), + ...GUIDANCE_VARIANTS.map((v) => ['metadata', v] as Item), + ...GUIDANCE_VARIANTS.map((v) => ['viewport', v] as Item), ['client-hook', 'runtime'], - ['metadata', 'runtime'], - ['metadata', 'dynamic'], - ['metadata', 'link'], - ['viewport', 'runtime'], - ['viewport', 'dynamic'], - ['viewport', 'link'], ['unrendered-segment', 'runtime'], ['link-prefetch-partial', 'runtime'], ] diff --git a/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.ts b/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.ts index f1fc4a6c323a..98fd4e2cc44d 100644 --- a/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.ts +++ b/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance-data.ts @@ -98,6 +98,33 @@ const linkCards: FixCard[] = [ }, ] +// TODO(cache-stages): docs link +const navigationCards: FixCard[] = [ + { + id: 'wrap-in-or-move-into-suspense', + title: 'Wrap in or move into Suspense', + group: 'stream', + link: 'https://nextjs.org/docs/messages/instant-shell-url-data#wrap-in-or-move-into-suspense', + snippets: [ + { text: '', highlight: true }, + { text: ' ' }, + { text: '', highlight: true }, + ], + copyable: true, + }, + { + id: 'allow-blocking-route', + title: 'Allow blocking route', + group: 'block', + link: 'https://nextjs.org/docs/messages/instant-shell-url-data#allow-blocking-route', + snippets: [ + { text: '// page.tsx or layout.tsx' }, + { text: 'export const instant = false', highlight: true }, + ], + copyable: true, + }, +] + const runtimeCards: FixCard[] = [ { id: 'wrap-in-or-move-into-suspense', @@ -293,6 +320,33 @@ const metadataRuntimeCards: FixCard[] = [ }, ] +// TODO(cache-stages): docs link +const metadataNavigationCards: FixCard[] = [ + { + id: 'use-static-metadata', + title: 'Use static metadata', + group: 'static', + link: 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime#use-static-metadata', + snippets: [ + { text: 'export const metadata = {', highlight: true }, + { text: ' title: "My Page"' }, + { text: '}' }, + ], + copyable: true, + }, + { + id: 'mark-the-route-as-dynamic', + title: 'Mark the route as dynamic', + group: 'dynamic', + link: 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime#mark-the-route-as-dynamic', + snippets: [ + { text: '// page.tsx or layout.tsx' }, + { text: 'await connection()', highlight: true }, + ], + copyable: true, + }, +] + // URL data in `generateMetadata()` shares the same fixes as runtime data. const metadataLinkCards = metadataRuntimeCards @@ -348,6 +402,33 @@ const viewportRuntimeCards: FixCard[] = [ }, ] +// TODO(cache-stages): docs link +const viewportNavigationCards: FixCard[] = [ + { + id: 'use-static-viewport', + title: 'Use static viewport', + group: 'static', + link: 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime#use-static-viewport', + snippets: [ + { text: 'export const viewport = {', highlight: true }, + { text: ' themeColor: "#000"' }, + { text: '}' }, + ], + copyable: true, + }, + { + id: 'allow-blocking-route', + title: 'Allow blocking route', + group: 'block', + link: 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime#allow-blocking-route', + snippets: [ + { text: '// page.tsx or layout.tsx' }, + { text: 'export const instant = false', highlight: true }, + ], + copyable: true, + }, +] + // URL data in `generateViewport()` shares the same fixes as runtime data. const viewportLinkCards = viewportRuntimeCards @@ -609,7 +690,7 @@ export type GuidanceKind = | 'unrendered-segment' | 'link-prefetch-partial' -export type GuidanceVariant = 'link' | 'runtime' | 'dynamic' +export type GuidanceVariant = 'link' | 'runtime' | 'navigation' | 'dynamic' export const DOCS_URLS: Record = { 'blocking-route': 'https://nextjs.org/docs/messages/blocking-route', @@ -627,6 +708,39 @@ export const DOCS_URLS: Record = { 'https://nextjs.org/docs/messages/instant-link-prefetch-partial', } +export const BLOCKING_ROUTE_DOCS_URLS: Record = { + runtime: 'https://nextjs.org/docs/messages/blocking-prerender-runtime', + // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) + link: 'https://nextjs.org/docs/messages/blocking-prerender-runtime', + // TODO(cache-stages): dedicated docs for navigation errors (reuses runtime for now) + navigation: 'https://nextjs.org/docs/messages/blocking-prerender-runtime', + dynamic: 'https://nextjs.org/docs/messages/blocking-prerender-dynamic', +} + +export const BLOCKING_METADATA_DOCS_URLS: Record = { + runtime: + 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime', + // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) + link: 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime', + // TODO(cache-stages): dedicated docs for navigation errors (reuses runtime for now) + navigation: + 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime', + dynamic: + 'https://nextjs.org/docs/messages/blocking-prerender-metadata-dynamic', +} + +export const BLOCKING_VIEWPORT_DOCS_URLS: Record = { + runtime: + 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime', + // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) + link: 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime', + // TODO(cache-stages): dedicated docs for navigation errors (reuses runtime for now) + navigation: + 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime', + dynamic: + 'https://nextjs.org/docs/messages/blocking-prerender-viewport-dynamic', +} + export const SYNC_IO_DOCS: Record = { 'Math.random()': 'https://nextjs.org/docs/messages/blocking-prerender-random', 'Date.now()': @@ -701,10 +815,10 @@ export const EXPLANATIONS: Record = { 'This will lead to slower, more expensive prefetches.', } -export const BLOCKING_ROUTE_NAVIGATION_EXPLANATION = +export const BLOCKING_ROUTE_IN_NAVIGATION_EXPLANATION = 'This prevents the navigation from being instant, leading to a slower user experience.' -export const BLOCKING_ROUTE_LINK_EXPLANATION = +export const BLOCKING_ROUTE_BLOCKED_SHELL_EXPLANATION = 'This may prevent the navigation from being instant, leading to a slower user experience.' const syncCardsByCause: Record = { @@ -755,26 +869,50 @@ export function getCards( cause?: string ): FixCard[] { switch (kind) { - case 'blocking-route': - return variant === 'link' - ? linkCards - : variant === 'dynamic' - ? filterCacheForConnection(dynamicCards, variant, cause) - : runtimeCards + case 'blocking-route': { + switch (variant) { + case 'link': + return linkCards + case 'runtime': + return runtimeCards + case 'navigation': + return navigationCards + case 'dynamic': + return filterCacheForConnection(dynamicCards, variant, cause) + default: + return variant satisfies never + } + } case 'client-hook': return clientHookCards - case 'metadata': - return variant === 'link' - ? metadataLinkCards - : variant === 'runtime' - ? metadataRuntimeCards - : filterCacheForConnection(metadataDynamicCards, variant, cause) - case 'viewport': - return variant === 'link' - ? viewportLinkCards - : variant === 'runtime' - ? viewportRuntimeCards - : filterCacheForConnection(viewportDynamicCards, variant, cause) + case 'metadata': { + switch (variant) { + case 'link': + return metadataLinkCards + case 'runtime': + return metadataRuntimeCards + case 'navigation': + return metadataNavigationCards + case 'dynamic': + return filterCacheForConnection(metadataDynamicCards, variant, cause) + default: + return variant satisfies never + } + } + case 'viewport': { + switch (variant) { + case 'link': + return viewportLinkCards + case 'runtime': + return viewportRuntimeCards + case 'navigation': + return viewportNavigationCards + case 'dynamic': + return filterCacheForConnection(viewportDynamicCards, variant, cause) + default: + return variant satisfies never + } + } case 'sync-io': return (cause && syncCardsByCause[cause]) || [] case 'sync-io-client': diff --git a/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance.tsx b/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance.tsx index d741a442a0a2..042281e4f98c 100644 --- a/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance.tsx +++ b/packages/next/src/next-devtools/dev-overlay/components/instant/instant-guidance.tsx @@ -17,6 +17,9 @@ import { ExternalIcon } from '../../icons/external' import { CopyPromptIcon } from '../../icons/copy-prompt' import { css } from '../../utils/css' import { + BLOCKING_METADATA_DOCS_URLS, + BLOCKING_ROUTE_DOCS_URLS, + BLOCKING_VIEWPORT_DOCS_URLS, DOCS_URLS, EXPLANATIONS, FIX_CARD_GROUPS, @@ -243,37 +246,30 @@ export function InstantGuidance({ }) { const cards = getCards(kind, variant, cause) let docsUrl: string - if (kind === 'sync-io' && cause) { - docsUrl = SYNC_IO_DOCS[cause] || DOCS_URLS[kind] - } else if (kind === 'sync-io-client' && cause) { - docsUrl = SYNC_IO_CLIENT_DOCS[cause] || DOCS_URLS[kind] - } else if (kind === 'blocking-route') { - docsUrl = - // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) - variant === 'link' - ? 'https://nextjs.org/docs/messages/blocking-prerender-runtime' - : variant === 'runtime' - ? 'https://nextjs.org/docs/messages/blocking-prerender-runtime' - : 'https://nextjs.org/docs/messages/blocking-prerender-dynamic' - } else if (kind === 'metadata') { - docsUrl = - // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) - variant === 'link' - ? 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime' - : variant === 'runtime' - ? 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime' - : 'https://nextjs.org/docs/messages/blocking-prerender-metadata-dynamic' - } else if (kind === 'viewport') { - docsUrl = - // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) - variant === 'link' - ? 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime' - : variant === 'runtime' - ? 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime' - : 'https://nextjs.org/docs/messages/blocking-prerender-viewport-dynamic' - } else { - docsUrl = DOCS_URLS[kind] + switch (kind) { + case 'sync-io': + docsUrl = (cause ? SYNC_IO_DOCS[cause] : undefined) ?? DOCS_URLS[kind] + break + case 'sync-io-client': + docsUrl = + (cause ? SYNC_IO_CLIENT_DOCS[cause] : undefined) ?? DOCS_URLS[kind] + break + case 'blocking-route': + docsUrl = BLOCKING_ROUTE_DOCS_URLS[variant] + break + case 'metadata': + docsUrl = BLOCKING_METADATA_DOCS_URLS[variant] + break + case 'viewport': + docsUrl = BLOCKING_VIEWPORT_DOCS_URLS[variant] + break + case 'client-hook': + case 'unrendered-segment': + case 'link-prefetch-partial': + docsUrl = DOCS_URLS[kind] + break } + const defaultExplanation = explanation || EXPLANATIONS[kind] return ( @@ -315,29 +311,11 @@ export function InstantHeaderExplanation({ const resolvedExplanation = explanation || (kind ? EXPLANATIONS[kind] : '') let resolvedDocsUrl = docsUrl if (!resolvedDocsUrl && kind === 'blocking-route') { - resolvedDocsUrl = - // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) - variant === 'link' - ? 'https://nextjs.org/docs/messages/blocking-prerender-runtime' - : variant === 'runtime' - ? 'https://nextjs.org/docs/messages/blocking-prerender-runtime' - : 'https://nextjs.org/docs/messages/blocking-prerender-dynamic' + resolvedDocsUrl = BLOCKING_ROUTE_DOCS_URLS[variant ?? 'dynamic'] } else if (!resolvedDocsUrl && kind === 'metadata') { - resolvedDocsUrl = - // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) - variant === 'link' - ? 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime' - : variant === 'runtime' - ? 'https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime' - : 'https://nextjs.org/docs/messages/blocking-prerender-metadata-dynamic' + resolvedDocsUrl = BLOCKING_METADATA_DOCS_URLS[variant ?? 'dynamic'] } else if (!resolvedDocsUrl && kind === 'viewport') { - resolvedDocsUrl = - // TODO(app-shells): dedicated docs for link data errors (reuses runtime for now) - variant === 'link' - ? 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime' - : variant === 'runtime' - ? 'https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime' - : 'https://nextjs.org/docs/messages/blocking-prerender-viewport-dynamic' + resolvedDocsUrl = BLOCKING_VIEWPORT_DOCS_URLS[variant ?? 'dynamic'] } else if (!resolvedDocsUrl && kind) { resolvedDocsUrl = DOCS_URLS[kind] } diff --git a/packages/next/src/next-devtools/dev-overlay/container/errors.test.ts b/packages/next/src/next-devtools/dev-overlay/container/errors.test.ts index 690f1b675af4..8ddfa3baced9 100644 --- a/packages/next/src/next-devtools/dev-overlay/container/errors.test.ts +++ b/packages/next/src/next-devtools/dev-overlay/container/errors.test.ts @@ -13,6 +13,9 @@ import { createLinkBodyErrorInNavigation, createLinkMetadataError, createLinkViewportError, + createNavigationBodyErrorInNavigation, + createNavigationMetadataError, + createNavigationViewportError, } from '../../../server/app-render/blocking-route-messages' import { createSyncIOClientError, @@ -37,25 +40,6 @@ const ROUTE = '/example' describe('getGuidanceVariant', () => { describe('classifies runtime messages as runtime', () => { - describe('classifies link messages as link', () => { - it.each([ - { - description: 'body', - error: () => createLinkBodyErrorInNavigation(ROUTE), - }, - { - description: 'metadata', - error: () => createLinkMetadataError(ROUTE), - }, - { - description: 'viewport', - error: () => createLinkViewportError(ROUTE), - }, - ])('$description', ({ error }) => { - expect(getGuidanceVariant(error().message)).toBe('link') - }) - }) - it.each([ { description: 'body', error: () => createRuntimeBodyError(ROUTE) }, { @@ -75,6 +59,44 @@ describe('getGuidanceVariant', () => { }) }) + describe('classifies link messages as link', () => { + it.each([ + { + description: 'body', + error: () => createLinkBodyErrorInNavigation(ROUTE), + }, + { + description: 'metadata', + error: () => createLinkMetadataError(ROUTE), + }, + { + description: 'viewport', + error: () => createLinkViewportError(ROUTE), + }, + ])('$description', ({ error }) => { + expect(getGuidanceVariant(error().message)).toBe('link') + }) + }) + + describe('classifies navigation messages as navigation', () => { + it.each([ + { + description: 'body', + error: () => createNavigationBodyErrorInNavigation(ROUTE), + }, + { + description: 'metadata', + error: () => createNavigationMetadataError(ROUTE), + }, + { + description: 'viewport', + error: () => createNavigationViewportError(ROUTE), + }, + ])('$description', ({ error }) => { + expect(getGuidanceVariant(error().message)).toBe('navigation') + }) + }) + describe('classifies dynamic messages as dynamic', () => { it.each([ { description: 'body', error: () => createDynamicBodyError(ROUTE) }, @@ -222,6 +244,16 @@ describe('getBlockingRouteErrorDetails', () => { }) }) + it('classifies createNavigationBodyErrorInNavigation as blocking-route + navigation + inNavigation', () => { + expect( + getBlockingRouteErrorDetails(createNavigationBodyErrorInNavigation(ROUTE)) + ).toEqual({ + type: 'blocking-route', + variant: 'navigation', + inNavigation: true, + }) + }) + it('classifies createDynamicOrRuntimeBodyError as blocking-route + dynamic (SSR-only)', () => { // The "either" factory has no clear runtime signal — falls into the // dynamic branch by `isRuntimeVariant`. Documents current behavior. @@ -240,6 +272,18 @@ describe('getBlockingRouteErrorDetails', () => { ).toEqual({ type: 'dynamic-metadata', variant: 'runtime' }) }) + it('classifies createLinkMetadataError as dynamic-metadata + link', () => { + expect( + getBlockingRouteErrorDetails(createLinkMetadataError(ROUTE)) + ).toEqual({ type: 'dynamic-metadata', variant: 'link' }) + }) + + it('classifies createNavigationMetadataError as dynamic-metadata + navigation', () => { + expect( + getBlockingRouteErrorDetails(createNavigationMetadataError(ROUTE)) + ).toEqual({ type: 'dynamic-metadata', variant: 'navigation' }) + }) + it('classifies createDynamicMetadataError as dynamic-metadata + dynamic', () => { expect( getBlockingRouteErrorDetails(createDynamicMetadataError(ROUTE)) @@ -258,6 +302,18 @@ describe('getBlockingRouteErrorDetails', () => { ).toEqual({ type: 'dynamic-viewport', variant: 'runtime' }) }) + it('classifies createLinkViewportError as dynamic-viewport + link', () => { + expect( + getBlockingRouteErrorDetails(createLinkViewportError(ROUTE)) + ).toEqual({ type: 'dynamic-viewport', variant: 'link' }) + }) + + it('classifies createNavigationViewportError as dynamic-viewport + navigation', () => { + expect( + getBlockingRouteErrorDetails(createNavigationViewportError(ROUTE)) + ).toEqual({ type: 'dynamic-viewport', variant: 'navigation' }) + }) + it('classifies createDynamicViewportError as dynamic-viewport + dynamic', () => { expect( getBlockingRouteErrorDetails(createDynamicViewportError(ROUTE)) diff --git a/packages/next/src/next-devtools/dev-overlay/container/errors.tsx b/packages/next/src/next-devtools/dev-overlay/container/errors.tsx index 3fd4d073de97..b8ec57aed21f 100644 --- a/packages/next/src/next-devtools/dev-overlay/container/errors.tsx +++ b/packages/next/src/next-devtools/dev-overlay/container/errors.tsx @@ -35,8 +35,8 @@ import { type GuidanceVariant, } from '../components/instant/instant-guidance' import { - BLOCKING_ROUTE_NAVIGATION_EXPLANATION, - BLOCKING_ROUTE_LINK_EXPLANATION, + BLOCKING_ROUTE_IN_NAVIGATION_EXPLANATION, + BLOCKING_ROUTE_BLOCKED_SHELL_EXPLANATION, } from '../components/instant/instant-guidance-data' import { UnrenderedSegmentInfo } from '../components/instant/unrendered-segment-info' import { CodeFrame } from '../components/code-frame/code-frame' @@ -353,9 +353,12 @@ function InstantRuntimeError({ } export function getGuidanceVariant(message: string): GuidanceVariant { - // Discriminates between `createLinkBodyErrorInNavigation`, - // `createRuntimeBodyError`, and `createDynamicBodyError` (and their - // in-navigation variants). + // Discriminates between `createNavigationBodyErrorInNavigation`, + // `createLinkBodyErrorInNavigation`, `createRuntimeBodyError`, and + // `createDynamicBodyError` (and their in-navigation variants). + if (message.includes('encountered `unstable_navigation()`')) { + return 'navigation' + } if ( message.includes('encountered URL data') && !message.includes('encountered uncached data') @@ -859,31 +862,47 @@ export function Errors({ ) } break - case 'blocking-route': + case 'blocking-route': { + switch (errorDetails.variant) { + case 'runtime': + errorMessage = errorDetails.inNavigation + ? 'Next.js encountered runtime data during a navigation.' + : 'Next.js encountered runtime data during prerendering.' + break + case 'link': + errorMessage = 'Next.js encountered URL data outside of Suspense.' + break + case 'navigation': + errorMessage = ( + <> + Next.js encountered unstable_navigation() outside of + Suspense. + + ) + break + case 'dynamic': + errorMessage = errorDetails.inNavigation + ? 'Next.js encountered uncached data during a navigation.' + : 'Next.js encountered uncached data during prerendering.' + break + default: + errorMessage = errorDetails.variant satisfies never + } return ( @@ -915,6 +934,7 @@ export function Errors({ ) + } case 'client-hook': return ( ) - case 'dynamic-metadata': + case 'dynamic-metadata': { + switch (errorDetails.variant) { + case 'runtime': + errorMessage = ( + <> + Next.js encountered runtime data in{' '} + generateMetadata(). + + ) + break + case 'link': + errorMessage = ( + <> + Next.js encountered URL data in generateMetadata(). + + ) + break + case 'navigation': + errorMessage = ( + <> + Next.js encountered unstable_navigation() in{' '} + generateMetadata(). + + ) + break + case 'dynamic': + errorMessage = ( + <> + Next.js encountered uncached data in{' '} + generateMetadata(). + + ) + break + default: + errorMessage = errorDetails.variant satisfies never + } return ( - Next.js encountered URL data in generateMetadata(). - - ) : errorDetails.variant === 'runtime' ? ( - <> - Next.js encountered runtime data in{' '} - generateMetadata(). - - ) : ( - <> - Next.js encountered uncached data in{' '} - generateMetadata(). - - ) - } + errorMessage={errorMessage} headerChildren={ ) - case 'dynamic-viewport': + } + case 'dynamic-viewport': { + switch (errorDetails.variant) { + case 'link': + errorMessage = ( + <> + Next.js encountered URL data in generateViewport(). + + ) + break + case 'runtime': + errorMessage = ( + <> + Next.js encountered runtime data in{' '} + generateViewport(). + + ) + break + case 'navigation': + errorMessage = ( + <> + Next.js encountered unstable_navigation() in{' '} + generateViewport(). + + ) + break + case 'dynamic': + errorMessage = ( + <> + Next.js encountered uncached data in{' '} + generateViewport(). + + ) + break + default: + errorMessage = errorDetails.variant satisfies never + } + return ( - Next.js encountered URL data in generateViewport(). - - ) : errorDetails.variant === 'runtime' ? ( - <> - Next.js encountered runtime data in{' '} - generateViewport(). - - ) : ( - <> - Next.js encountered uncached data in{' '} - generateViewport(). - - ) - } + errorMessage={errorMessage} headerChildren={ ) + } case 'sync-io': return ( + } + const validationSequences = { + [ValidationPrefetchKind.Shell]: { + stageOrder: [ + RenderStage.ShellRuntime, + RenderStage.Runtime, + RenderStage.NavigationRuntime, + RenderStage.Dynamic, + ], + holeResolution: { + [RenderStage.Static]: null, // no holes resolve in the Static stage (URL data like static params goes in the Runtime stage) + [RenderStage.ShellRuntime]: null, // initial stage + [RenderStage.Runtime]: DynamicHoleKind.Link, + [RenderStage.NavigationRuntime]: DynamicHoleKind.Navigation, + [RenderStage.Dynamic]: DynamicHoleKind.Dynamic, + }, + } as ValidationSequence, + [ValidationPrefetchKind.LegacySpeculative]: { + stageOrder: [ + RenderStage.Static, + RenderStage.Runtime, + RenderStage.Dynamic, + ], + holeResolution: { + [RenderStage.Static]: null, // initial stage + [RenderStage.ShellRuntime]: null, // currently unused in static prefetch validation. + [RenderStage.Runtime]: DynamicHoleKind.Runtime, + [RenderStage.NavigationRuntime]: null, // static prefetches never have navigation() holes. + [RenderStage.Dynamic]: DynamicHoleKind.Dynamic, + }, + } as ValidationSequence, + } as const + + const validationSequence = validationSequences[prefetchKind] + const initialRenderStage = validationSequence + .stageOrder[0] as PrefetchedSegmentStage + // When narrowing the cause of a dynamic hole, we need to find the first stage it occurs in. + // We do this by starting at the end and eliminating later stages, so we go in reverse. + const retryRenderStages = validationSequences[prefetchKind].stageOrder + .slice(1, -1) // Exclude first stage and Dynamic + .reverse() as RetryStage[] + + function getDynamicHoleKindForSegmentStage( + stage: PrefetchedSegmentStage + ): DynamicHoleKind { + // We report holes in reverse order, i.e. holes in Stage N are only reported + // if Stage N+1 didn't have any holes. That means that if we report a hole from Stage N, + // it has to be caused by data that would've resolved in Stage N+1. + // So, the dynamic hole kind corresponds to the *next* logical stage. + const { stageOrder, holeResolution } = validationSequence + const nextStage = stageOrder[stageOrder.indexOf(stage) + 1] + const holeKind = holeResolution[nextStage] + if (!holeKind) { + throw new InvariantError( + `${RenderStage[stage]} segments do not unblock new data in ${ValidationPrefetchKind[prefetchKind]} prefetches` + ) + } + return holeKind + } + async function validateAtDepth( depth: number, groupDepthForValidation: number ): Promise { - return validateAtDepthImpl(depth, groupDepthForValidation, null) + if (validationAbortSignal?.aborted) { + return null + } + + // First attempt. If we have any dynamic holes, they will be ambiguous. + const initialBoundaryState = createValidationBoundaryTracking() + const initialResult = await validateOrRetryAtDepth( + depth, + groupDepthForValidation, + initialBoundaryState, + null + ) + + // If the prerender produced no real errors at this depth — either an + // empty array (clean) or a deferred-only result (Error/AggregateError + // representing a missing-boundary fallback) — there's nothing to + // discriminate. Pass it up so the outer loop can hold any deferred + // fallback back until every depth has been tried. + if (!Array.isArray(initialResult) || initialResult.length === 0) { + return initialResult + } + + // We do a followup validation using a payload using later stages to determine + // what kind of data caused the hole -- link/navigation/dynamic in app shells, + // or runtime/dynamic in static shells. + for (const retryStage of retryRenderStages) { + if ( + validationAbortSignal !== undefined && + !(await yieldToForegroundRequest(validationAbortSignal)) + ) { + return [] + } + + const narrowedResult = await validateOrRetryAtDepth( + depth, + groupDepthForValidation, + createValidationBoundaryTracking(initialBoundaryState), + retryStage + ) + + if (Array.isArray(narrowedResult) && narrowedResult.length > 0) { + // The narrowed validation found errors to report. + return narrowedResult + } + } + + // If we didn't return some other errors at this point the only thing to return is this validation's result + return initialResult } - async function validateAtDepthImpl( + async function validateOrRetryAtDepth( depth: number, groupDepthForValidation: number, - previousBoundaryState: null | ValidationBoundaryTracking - ): Promise { - if (validationAbortSignal?.aborted) { - return null - } + boundaryState: ValidationBoundaryTracking, + overrideStageForPartialSegments: RetryStage | null = null + ): Promise { + // If we're not overriding the stage, we're in the first stage. + const stage = overrideStageForPartialSegments ?? initialRenderStage + const dynamicHoleKind = getDynamicHoleKindForSegmentStage(stage) const extraChunksController = new AbortController() const extraChunksSignal = @@ -7387,16 +7507,6 @@ async function validateInstantConfigs( ? extraChunksController.signal : AbortSignal.any([extraChunksController.signal, validationAbortSignal]) - const boundaryState = createValidationBoundaryTracking() - let useRuntimeStageForPartialSegments = false - if (previousBoundaryState) { - // We're doing a followup render to better discriminate error types - useRuntimeStageForPartialSegments = true - for (const [id, filePath] of previousBoundaryState.requiredIds) { - boundaryState.requiredIds.set(id, filePath) - } - } - const payloadResult = await createCombinedPayloadAtDepth( prefetchKind, initialRscPayload, @@ -7409,7 +7519,7 @@ async function validateInstantConfigs( extraChunksSignal, boundaryState, clientReferenceManifest, - useRuntimeStageForPartialSegments + overrideStageForPartialSegments ) if (payloadResult === null) { @@ -7469,23 +7579,6 @@ async function validateInstantConfigs( validationSampleTracking, } - let dynamicHoleKind: DynamicHoleKind - switch (prefetchKind) { - case ValidationPrefetchKind.Shell: { - dynamicHoleKind = payloadResult.hasAmbiguousErrors - ? DynamicHoleKind.Link - : DynamicHoleKind.Dynamic - break - } - case ValidationPrefetchKind.LegacySpeculative: { - dynamicHoleKind = payloadResult.hasAmbiguousErrors - ? DynamicHoleKind.Runtime - : DynamicHoleKind.Dynamic - break - } - } - - let result: NavigationValidationResult try { const { prelude: unprocessedPrelude } = await runInSequentialTasks( () => { @@ -7580,7 +7673,7 @@ async function validateInstantConfigs( const { preludeIsEmpty } = await processPreludeOp(unprocessedPrelude) - result = getNavigationDisallowedDynamicReasons( + return getNavigationDisallowedDynamicReasons( workStore, preludeIsEmpty ? PreludeState.Empty : PreludeState.Full, instantValidationState, @@ -7589,7 +7682,7 @@ async function validateInstantConfigs( devRenderDidError ) } catch (thrownValue) { - result = getNavigationDisallowedDynamicReasons( + return getNavigationDisallowedDynamicReasons( workStore, PreludeState.Errored, instantValidationState, @@ -7598,40 +7691,6 @@ async function validateInstantConfigs( devRenderDidError ) } - - // If the prerender produced no real errors at this depth — either an - // empty array (clean) or a deferred-only result (Error/AggregateError - // representing a missing-boundary fallback) — there's nothing to - // discriminate. Pass it up so the outer loop can hold any deferred - // fallback back until every depth has been tried. - if (!Array.isArray(result) || result.length === 0) { - return result - } - - if (previousBoundaryState === null && payloadResult.hasAmbiguousErrors) { - // This is the first validation attempt. we prepared a payload where dynamic holes might be runtime data dependencies - // or dynamic data dependencies. We do a followup validation using a payload with only Runtime segments to discriminate - if ( - validationAbortSignal !== undefined && - !(await yieldToForegroundRequest(validationAbortSignal)) - ) { - return [] - } - - const dynamicOnlyResult = await validateAtDepthImpl( - depth, - groupDepthForValidation, - boundaryState - ) - - if (Array.isArray(dynamicOnlyResult) && dynamicOnlyResult.length > 0) { - // The dynamic errors only validation found errors to report so we favor those - return dynamicOnlyResult - } - } - - // If we didn't return some other errors at this point the only thing to return is this validation's result - return result } // Discover validation depth bounds from the LoaderTree. The array diff --git a/packages/next/src/server/app-render/blocking-route-messages.ts b/packages/next/src/server/app-render/blocking-route-messages.ts index ea60a1fb0e48..941fa1569c56 100644 --- a/packages/next/src/server/app-render/blocking-route-messages.ts +++ b/packages/next/src/server/app-render/blocking-route-messages.ts @@ -43,6 +43,18 @@ export function createLinkBodyErrorInNavigation(route: string): Error { ) } +export function createNavigationBodyErrorInNavigation(route: string): Error { + // TODO(cache-stages): docs link + return new Error( + `Route "${route}": Next.js encountered \`unstable_navigation()\` during prerendering or a navigation.\n\n` + + `\`unstable_navigation()\` called outside of \`\` may prevent the navigation from being instant, leading to a slower user experience.\n\n` + + `Ways to fix this:\n` + + ` - [stream] Provide a placeholder with \`\` around the data access\n` + + ` - [block] Set \`export const instant = false\` to allow a blocking route\n\n` + + `Learn more: https://nextjs.org/docs/messages/instant-shell-url-data` + ) +} + export function createDynamicBodyErrorInNavigation(route: string): Error { return new Error( `Route "${route}": Next.js encountered uncached data during prerendering or a navigation.\n\n` + @@ -94,6 +106,18 @@ export function createRuntimeMetadataError(route: string): Error { ) } +export function createNavigationMetadataError(route: string): Error { + // TODO(cache-stages): docs link + return new Error( + `Route "${route}": Next.js encountered \`unstable_navigation()\` in \`generateMetadata()\`.\n\n` + + `This route's metadata is blocked, but the rest of its content can be prefetched. \`unstable_navigation()\` called in \`generateMetadata()\` prevents it from being prefetched.\n\n` + + `Ways to fix this:\n` + + ` - [static] Use a static metadata export instead of \`generateMetadata()\`\n` + + ` - [dynamic] Render a marker component that calls \`await connection()\` inside \`\` on the page\n\n` + + `Learn more: https://nextjs.org/docs/messages/blocking-prerender-metadata-runtime` + ) +} + export function createDynamicMetadataError(route: string): Error { return new Error( `Route "${route}": Next.js encountered uncached data in \`generateMetadata()\`.\n\n` + @@ -127,6 +151,18 @@ export function createRuntimeViewportError(route: string): Error { ) } +export function createNavigationViewportError(route: string): Error { + // TODO(cache-stages): docs link + return new Error( + `Route "${route}": Next.js encountered \`unstable_navigation()\` in \`generateViewport()\`.\n\n` + + `\`unstable_navigation()\` in \`generateViewport()\` prevents creating a shell, leading to a slower user experience.\n\n` + + `Ways to fix this:\n` + + ` - [static] Use a static viewport export instead of \`generateViewport()\`\n` + + ` - [block] Set \`export const instant = false\` to allow a blocking route\n\n` + + `Learn more: https://nextjs.org/docs/messages/blocking-prerender-viewport-runtime` + ) +} + export function createDynamicViewportError(route: string): Error { return new Error( `Route "${route}": Next.js encountered uncached data in \`generateViewport()\`.\n\n` + diff --git a/packages/next/src/server/app-render/dev-validation-scheduler.ts b/packages/next/src/server/app-render/dev-validation-scheduler.ts index 3b8b84ca46d1..5484e1f7c356 100644 --- a/packages/next/src/server/app-render/dev-validation-scheduler.ts +++ b/packages/next/src/server/app-render/dev-validation-scheduler.ts @@ -79,9 +79,9 @@ export function beginDevValidation( * Give incoming requests a chance to enter app rendering and supersede the * current validation before another expensive render attempt starts. * - * The regular global `setImmediate` is patched by staged rendering and can run - * inside the current timer task. The original immediate is required here so we - * actually pass through the event-loop poll phase where HTTP requests arrive. + * @returns Whether validation should continue. + * - `true`: validation should continue + * - `false` the validation was superseded and should be aborted. */ export async function yieldToForegroundRequest( validationSignal: AbortSignal @@ -90,6 +90,9 @@ export async function yieldToForegroundRequest( return false } + // Pass through the event-loop poll phase where HTTP requests arrive. + // Defensively use the unpatched setImmediate (even though this should never run + // in a context where setImmediate is patched, e.g. in `runInSequentialTasks`) await new Promise((resolve) => unpatchedSetImmediate(resolve)) return !validationSignal.aborted } diff --git a/packages/next/src/server/app-render/dynamic-rendering.ts b/packages/next/src/server/app-render/dynamic-rendering.ts index d108561aa849..56c0a172e864 100644 --- a/packages/next/src/server/app-render/dynamic-rendering.ts +++ b/packages/next/src/server/app-render/dynamic-rendering.ts @@ -51,11 +51,13 @@ import { createRuntimeBodyError, createDynamicBodyError, createRuntimeBodyErrorInNavigation, + createNavigationBodyErrorInNavigation, createDynamicBodyErrorInNavigation, createDynamicOrRuntimeBodyError, createRuntimeMetadataError, createDynamicMetadataError, createRuntimeViewportError, + createNavigationViewportError, createDynamicViewportError, createDynamicOrRuntimeViewportError, createDynamicOrRuntimeMetadataError, @@ -63,6 +65,7 @@ import { createLinkBodyErrorInNavigation, createLinkMetadataError, createLinkViewportError, + createNavigationMetadataError, } from './blocking-route-messages' import { InvariantError } from '../../shared/lib/invariant-error' import { @@ -696,12 +699,14 @@ export function trackAllowedDynamicAccess( } export enum DynamicHoleKind { - /** We know that this hole is caused by link data. */ - Link = 1, /** We know that this hole is caused by runtime data. */ - Runtime = 2, + Runtime = 1, + /** We know that this hole is caused by link data. */ + Link = 2, + /** We know that this hole is caused by navigation(). */ + Navigation = 3, /** We know that this hole is caused by dynamic data. */ - Dynamic = 3, + Dynamic = 4, } /** Stores dynamic reasons used during an SSR render in instant validation. */ @@ -760,11 +765,7 @@ export function trackDynamicHoleInNavigation( if (hasMetadataRegex.test(componentStack)) { const error = addErrorContext( - kind === DynamicHoleKind.Link - ? createLinkMetadataError(workStore.route) - : kind === DynamicHoleKind.Runtime - ? createRuntimeMetadataError(workStore.route) - : createDynamicMetadataError(workStore.route), + createMetadataError(kind, workStore.route), componentStack, effectiveCreateInstantStack ) @@ -773,11 +774,7 @@ export function trackDynamicHoleInNavigation( } if (hasViewportRegex.test(componentStack)) { const error = addErrorContext( - kind === DynamicHoleKind.Link - ? createLinkViewportError(workStore.route) - : kind === DynamicHoleKind.Runtime - ? createRuntimeViewportError(workStore.route) - : createDynamicViewportError(workStore.route), + createViewportError(kind, workStore.route), componentStack, effectiveCreateInstantStack ) @@ -869,11 +866,7 @@ export function trackDynamicHoleInNavigation( } const error = addErrorContext( - kind === DynamicHoleKind.Link - ? createLinkBodyErrorInNavigation(workStore.route) - : kind === DynamicHoleKind.Runtime - ? createRuntimeBodyErrorInNavigation(workStore.route) - : createDynamicBodyErrorInNavigation(workStore.route), + createBodyErrorInNavigation(kind, workStore.route), componentStack, effectiveCreateInstantStack ) @@ -881,6 +874,48 @@ export function trackDynamicHoleInNavigation( return } +function createBodyErrorInNavigation( + kind: DynamicHoleKind, + route: string +): Error { + switch (kind) { + case DynamicHoleKind.Runtime: + return createRuntimeBodyErrorInNavigation(route) + case DynamicHoleKind.Link: + return createLinkBodyErrorInNavigation(route) + case DynamicHoleKind.Navigation: + return createNavigationBodyErrorInNavigation(route) + case DynamicHoleKind.Dynamic: + return createDynamicBodyErrorInNavigation(route) + } +} + +function createMetadataError(kind: DynamicHoleKind, route: string): Error { + switch (kind) { + case DynamicHoleKind.Runtime: + return createRuntimeMetadataError(route) + case DynamicHoleKind.Link: + return createLinkMetadataError(route) + case DynamicHoleKind.Navigation: + return createNavigationMetadataError(route) + case DynamicHoleKind.Dynamic: + return createDynamicMetadataError(route) + } +} + +function createViewportError(kind: DynamicHoleKind, route: string): Error { + switch (kind) { + case DynamicHoleKind.Runtime: + return createRuntimeViewportError(route) + case DynamicHoleKind.Link: + return createLinkViewportError(route) + case DynamicHoleKind.Navigation: + return createNavigationViewportError(route) + case DynamicHoleKind.Dynamic: + return createDynamicViewportError(route) + } +} + export function trackThrownErrorInNavigation( workStore: WorkStore, dynamicValidation: InstantValidationState, diff --git a/packages/next/src/server/app-render/instant-validation/boundary-tracking.tsx b/packages/next/src/server/app-render/instant-validation/boundary-tracking.tsx index 3ac6f51c5417..09e242d0f321 100644 --- a/packages/next/src/server/app-render/instant-validation/boundary-tracking.tsx +++ b/packages/next/src/server/app-render/instant-validation/boundary-tracking.tsx @@ -10,9 +10,12 @@ export type ValidationBoundaryTracking = { renderedIds: Set } -export function createValidationBoundaryTracking(): ValidationBoundaryTracking { +export function createValidationBoundaryTracking( + /** Pass if the render is expected to render the same IDs as a previous one. */ + matchPrevious?: ValidationBoundaryTracking +): ValidationBoundaryTracking { return { - requiredIds: new Map(), + requiredIds: matchPrevious ? new Map(matchPrevious.requiredIds) : new Map(), renderedIds: new Set(), } } diff --git a/packages/next/src/server/app-render/instant-validation/instant-validation.tsx b/packages/next/src/server/app-render/instant-validation/instant-validation.tsx index a7b771ae472b..1e3cc08a76e9 100644 --- a/packages/next/src/server/app-render/instant-validation/instant-validation.tsx +++ b/packages/next/src/server/app-render/instant-validation/instant-validation.tsx @@ -170,10 +170,11 @@ export type SegmentStage = | RenderStage.Static | RenderStage.ShellRuntime | RenderStage.Runtime + | RenderStage.NavigationRuntime | RenderStage.Dynamic /** The stages that a prefetched segment can be in. */ -type PrefetchedSegmentStage = Exclude +export type PrefetchedSegmentStage = Exclude export type StageChunks = Record @@ -206,7 +207,11 @@ export async function collectStagedSegmentData( let partialStages: SegmentStage[] switch (prefetchKind) { case ValidationPrefetchKind.Shell: { - partialStages = [RenderStage.ShellRuntime, RenderStage.Runtime] + partialStages = [ + RenderStage.ShellRuntime, + RenderStage.Runtime, + RenderStage.NavigationRuntime, // TODO(cache-stages): only if needed + ] break } case ValidationPrefetchKind.LegacySpeculative: { @@ -282,6 +287,7 @@ async function collectSegmentDataForStage( return 'Prerender' case RenderStage.ShellRuntime: // TODO(app-shells) - proper environmentName case RenderStage.Runtime: + case RenderStage.NavigationRuntime: return 'Prefetch' case RenderStage.Dynamic: return 'Server' @@ -792,6 +798,7 @@ function createSegmentCacheItem(): SegmentCacheItem { [RenderStage.Static]: null, [RenderStage.ShellRuntime]: null, [RenderStage.Runtime]: null, + [RenderStage.NavigationRuntime]: null, [RenderStage.Dynamic]: null, } } @@ -983,11 +990,6 @@ export function discoverValidationDepths(loaderTree: LoaderTree): number[] { */ export type ValidationPayloadResult = { payload: InitialRSCPayload - /** Whether errors from this payload could be ambiguous between runtime - * API access (cookies, headers) and uncached IO (connection, fetch). - * True when some segments used Static stage. False when all segments - * used Runtime stage and errors are definitively from uncached IO. */ - hasAmbiguousErrors: boolean /** Per-slot config factories indexed by slot marker index. When a * boundary spans multiple parallel slots, each slot gets a marker * component in the tree. The marker's index maps to this array to @@ -1016,7 +1018,10 @@ export async function createCombinedPayloadAtDepth( releaseSignal: AbortSignal, boundaryState: ValidationBoundaryTracking, clientReferenceManifest: ClientReferenceManifest, - useRuntimeStageForPartialSegments: boolean + overrideStageForPartialSegments: + | null + | RenderStage.Runtime + | RenderStage.NavigationRuntime ): Promise { const workStore = workAsyncStorage.getStore() if (!workStore) { @@ -1026,9 +1031,6 @@ export async function createCombinedPayloadAtDepth( } const { validationLevel, route } = workStore - let hasStaticSegments = false - let hasRuntimeSegments = false - // Index 0 is reserved for the root config. Slot markers start at 1. const slotStacks: Array<(() => Error) | null> = [null] @@ -1305,39 +1307,14 @@ export async function createCombinedPayloadAtDepth( } let stage: PrefetchedSegmentStage - switch (prefetchKind) { case ValidationPrefetchKind.Shell: { - if (useRuntimeStageForPartialSegments) { - stage = RenderStage.Runtime - } else { - stage = RenderStage.ShellRuntime - } - // We do not track `has{Static,Runtime}Segments` because they do not - // affect shell prefetches. + stage = overrideStageForPartialSegments ?? RenderStage.ShellRuntime break } case ValidationPrefetchKind.LegacySpeculative: { - if (useRuntimeStageForPartialSegments) { - stage = RenderStage.Runtime - } else { - // In legacy speculative prefetches, we always use static. - stage = RenderStage.Static - } - break - } - } - - switch (stage) { - case RenderStage.Static: { - hasStaticSegments = true - break - } - case RenderStage.ShellRuntime: { - break - } - case RenderStage.Runtime: { - hasRuntimeSegments = true + // In legacy speculative prefetches, we always use static. + stage = overrideStageForPartialSegments ?? RenderStage.Static break } } @@ -1455,38 +1432,16 @@ export async function createCombinedPayloadAtDepth( let headStage: PrefetchedSegmentStage switch (prefetchKind) { case ValidationPrefetchKind.Shell: { - if (useRuntimeStageForPartialSegments) { - headStage = RenderStage.Runtime - } else { - headStage = RenderStage.ShellRuntime - } + headStage = overrideStageForPartialSegments ?? RenderStage.ShellRuntime break } case ValidationPrefetchKind.LegacySpeculative: { - headStage = hasRuntimeSegments ? RenderStage.Runtime : RenderStage.Static + headStage = overrideStageForPartialSegments ?? RenderStage.Static break } } debug?.(` /_head - ${RenderStage[headStage]}`) - let hasAmbiguousErrors: boolean - switch (prefetchKind) { - case ValidationPrefetchKind.Shell: { - // In a shell prefetch, holes are always ambiguous - // (they can be either link data or dynamic data) - // unless we're already overriding and using the runtime stage, - // which resolves link data. - hasAmbiguousErrors = !useRuntimeStageForPartialSegments - break - } - case ValidationPrefetchKind.LegacySpeculative: { - // In the old prefetching mechanism, holes in static segments are ambiguous - // (they can be either runtime data or dynamic data). - hasAmbiguousErrors = hasStaticSegments - break - } - } - const head = await createValidationHead( cache, releaseSignal, @@ -1508,7 +1463,6 @@ export async function createCombinedPayloadAtDepth( return { payload, - hasAmbiguousErrors, slotStacks, } } diff --git a/test/e2e/app-dir/instant-validation/head-and-reporting.util.ts b/test/e2e/app-dir/instant-validation/head-and-reporting.util.ts index c6efa5031743..39f713a35f37 100644 --- a/test/e2e/app-dir/instant-validation/head-and-reporting.util.ts +++ b/test/e2e/app-dir/instant-validation/head-and-reporting.util.ts @@ -623,32 +623,32 @@ export function registerHeadAndReportingTests( if (isNextDev) { const browser = await navigateTo('/shells/invalid-runtime-params/123') await expect(browser).toDisplayCollapsedRedbox(` - { - "cause": [ - { - "label": "Caused by: Instant Validation", - "source": "app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (3:33) @ instant - > 3 | export const instant: Instant = { - | ^", - "stack": [ - "instant app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (3:33)", - "Set.forEach ", - ], - }, - ], - "code": "E1439", - "description": "Next.js encountered URL data outside of Suspense.", - "environmentLabel": "Server", - "label": "Instant", - "source": "app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (28:3) @ LinkData - > 28 | await params - | ^", - "stack": [ - "LinkData app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (28:3)", - "Page app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (22:7)", - ], - } - `) + { + "cause": [ + { + "label": "Caused by: Instant Validation", + "source": "app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (3:33) @ instant + > 3 | export const instant: Instant = { + | ^", + "stack": [ + "instant app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (3:33)", + "Set.forEach ", + ], + }, + ], + "code": "E1439", + "description": "Next.js encountered URL data outside of Suspense.", + "environmentLabel": "Server", + "label": "Instant", + "source": "app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (28:3) @ LinkData + > 28 | await params + | ^", + "stack": [ + "LinkData app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (28:3)", + "Page app/shells/(default)/invalid-runtime-params/[slug]/page.tsx (22:7)", + ], + } + `) } else { const result = await prerender( '/shells/(default)/invalid-runtime-params/[slug]' @@ -729,32 +729,32 @@ export function registerHeadAndReportingTests( `) } else { await expect(browser).toDisplayCollapsedRedbox(` - { - "cause": [ - { - "label": "Caused by: Instant Validation", - "source": "app/shells/(default)/invalid-runtime-searchparams/page.tsx (3:33) @ instant - > 3 | export const instant: Instant = { - | ^", - "stack": [ - "instant app/shells/(default)/invalid-runtime-searchparams/page.tsx (3:33)", - "Set.forEach ", - ], - }, - ], - "code": "E1439", - "description": "Next.js encountered URL data outside of Suspense.", - "environmentLabel": "Server", - "label": "Instant", - "source": "app/shells/(default)/invalid-runtime-searchparams/page.tsx (27:3) @ LinkData - > 27 | await searchParams - | ^", - "stack": [ - "LinkData app/shells/(default)/invalid-runtime-searchparams/page.tsx (27:3)", - "Page app/shells/(default)/invalid-runtime-searchparams/page.tsx (17:7)", - ], - } - `) + { + "cause": [ + { + "label": "Caused by: Instant Validation", + "source": "app/shells/(default)/invalid-runtime-searchparams/page.tsx (3:33) @ instant + > 3 | export const instant: Instant = { + | ^", + "stack": [ + "instant app/shells/(default)/invalid-runtime-searchparams/page.tsx (3:33)", + "Set.forEach ", + ], + }, + ], + "code": "E1439", + "description": "Next.js encountered URL data outside of Suspense.", + "environmentLabel": "Server", + "label": "Instant", + "source": "app/shells/(default)/invalid-runtime-searchparams/page.tsx (27:3) @ LinkData + > 27 | await searchParams + | ^", + "stack": [ + "LinkData app/shells/(default)/invalid-runtime-searchparams/page.tsx (27:3)", + "Page app/shells/(default)/invalid-runtime-searchparams/page.tsx (17:7)", + ], + } + `) } } else { const result = await prerender( @@ -840,32 +840,32 @@ export function registerHeadAndReportingTests( '/shells/invalid-static-with-gsp/123' ) await expect(browser).toDisplayCollapsedRedbox(` - { - "cause": [ - { - "label": "Caused by: Instant Validation", - "source": "app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (3:33) @ instant - > 3 | export const instant: Instant = { - | ^", - "stack": [ - "instant app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (3:33)", - "Set.forEach ", - ], - }, - ], - "code": "E1439", - "description": "Next.js encountered URL data outside of Suspense.", - "environmentLabel": "Server", - "label": "Instant", - "source": "app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (31:20) @ LinkData - > 31 | const { slug } = await params - | ^", - "stack": [ - "LinkData app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (31:20)", - "Page app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (25:7)", - ], - } - `) + { + "cause": [ + { + "label": "Caused by: Instant Validation", + "source": "app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (3:33) @ instant + > 3 | export const instant: Instant = { + | ^", + "stack": [ + "instant app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (3:33)", + "Set.forEach ", + ], + }, + ], + "code": "E1439", + "description": "Next.js encountered URL data outside of Suspense.", + "environmentLabel": "Server", + "label": "Instant", + "source": "app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (31:20) @ LinkData + > 31 | const { slug } = await params + | ^", + "stack": [ + "LinkData app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (31:20)", + "Page app/shells/(default)/invalid-static-with-gsp/[slug]/page.tsx (25:7)", + ], + } + `) } else { const result = await prerender( '/shells/(default)/invalid-static-with-gsp/[slug]' @@ -895,7 +895,6 @@ export function registerHeadAndReportingTests( }) it('invalid - unguarded navigation() in a shell', async () => { - // TODO(cache-stages): navigation() should not be reported as uncached data. if (isNextDev) { const browser = await navigateTo( '/shells/invalid-navigation-without-suspense' @@ -914,8 +913,8 @@ export function registerHeadAndReportingTests( ], }, ], - "code": "E1437", - "description": "Next.js encountered uncached data during a navigation.", + "code": "E1495", + "description": "Next.js encountered unstable_navigation() outside of Suspense.", "environmentLabel": "Server", "label": "Instant", "source": "app/shells/(default)/invalid-navigation-without-suspense/page.tsx (23:19) @ NavigationContent @@ -933,16 +932,15 @@ export function registerHeadAndReportingTests( ) expect(extractBuildValidationError(result.cliOutput)) .toMatchInlineSnapshot(` - "Error: Route "/shells/invalid-navigation-without-suspense": Next.js encountered uncached data during prerendering or a navigation. + "Error: Route "/shells/invalid-navigation-without-suspense": Next.js encountered \`unstable_navigation()\` during prerendering or a navigation. - \`fetch(...)\` or \`connection()\` accessed outside of \`\` prevents the route from being prerendered or the navigation from being instant, leading to a slower user experience. + \`unstable_navigation()\` called outside of \`\` may prevent the navigation from being instant, leading to a slower user experience. Ways to fix this: - [stream] Provide a placeholder with \`\` around the data access - - [cache] Cache the data access with \`"use cache"\` (does not apply to \`connection()\`) - [block] Set \`export const instant = false\` to allow a blocking route - Learn more: https://nextjs.org/docs/messages/blocking-prerender-dynamic + Learn more: https://nextjs.org/docs/messages/instant-shell-url-data at main () at body () at html ()