feat(diagnostics): say what a shutdown that ran out of budget was still holding

CHANGELOG.md

@@ -6,6 +6,39 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 From 1.0.0 onward this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 Pre-1.0 releases followed it in spirit; their breaking changes are marked **Breaking**.
 
+## [Unreleased]
+
+### Added
+
+- **A shutdown that runs out of budget now says what was holding it.** The
+  `application.shutdownTimeout` diagnostic carried a phase name and nothing
+  else, which left the only question that matters unanswered: which hosted
+  service, which operation, which scope. Its `details` now carry the phase, the
+  budget, how long it waited, the hosted service inside its own `stop`, the
+  services still up, the operations that never settled and the resource scope
+  tree — ids, names and counts, never an argument or a payload.
+
+- **`createTestHost().inspect()` says what a failed leak assertion could not.**
+  `leaks()` reports three counts; when one is not zero the next question is
+  which module or operation still holds something, and there was no way to ask.
+  `inspect()` answers it: the scope trees, the hosted services still up, the
+  operations that never settled.
+
+  `leaks()` itself is deliberately unchanged. Adding the fields there was the
+  obvious move and it broke the first extension it was tried on: the guide
+  tells you to assert on the whole object, `toEqual` sees a new field, and the
+  test fails for a reason that has nothing to do with the extension. A separate
+  method costs one call and breaks nobody.
+
+- **`RegistrationScope` and `ResourceScope` gained `inspect()`**, returning a
+  `ScopeInspection` — name, entry count, attached children. This is what both
+  of the above are built on, and it is safe to call at any point, including
+  during disposal.
+
+- **`onDiagnostic` is documented.** It has been part of `defineExtension` since
+  3.0.0 and appeared in no guide; the guide now has a Diagnostics section
+  listing the events and what they are useful for.
+
 ## [4.0.1] - 2026-08-29
 
 **A patch to the tree-view adapter, plus four corrections to what the project

README.md

@@ -278,7 +278,7 @@ Stated plainly, because a framework that is vague about its boundaries gets
 trusted for things it cannot do.
 
 - **Nothing unwinds on a crash**: If the extension host is killed, `deactivate` never runs — persist what matters when the operation that produced it completes, not during shutdown
-- **The shutdown budget is shared and hard**: VS Code races _every_ extension's deactivation against 5 seconds and then exits; the framework's own budget (3 s by default) sits inside that, and past it pending work is abandoned rather than awaited
+- **The shutdown budget is shared and hard**: VS Code races _every_ extension's deactivation against 5 seconds and then exits; the framework's own budget (3 s by default) sits inside that, and past it pending work is abandoned rather than awaited — the `application.shutdownTimeout` diagnostic names what was still holding on
 - **Rollback covers what the framework owns**: registrations, the services it created, resources placed in one of its scopes, started hosted services — it cannot un-write a file or un-send a request
 - **Leak detection has the same boundary**: it sees what the framework tracks, and nothing else
 - **Cancellation is cooperative**: aborting a signal asks a handler to stop; one that ignores its signal keeps running, and the framework cannot terminate it

docs/guide.md

@@ -36,6 +36,7 @@ same code runs in a test and in the editor.
 - [UI](#ui)
 - [Views: trees and webviews](#views-trees-and-webviews)
 - [Testing](#testing)
+- [Diagnostics](#diagnostics)
 - [Keeping package.json honest](#keeping-packagejson-honest)
 - [The escape hatch](#the-escape-hatch)
 - [Publishing an API](#publishing-an-api)
@@ -644,6 +645,71 @@ The Test Host does not reproduce VS Code. Anything that depends on what VS Code
 does _with_ what you hand it — rather than on what it hands back — needs a real
 Extension Host test.
 
+`host.leaks()` reports what the framework still owns after `stop()` — three
+counts, and the assertion worth writing at the end of every host test. When one
+of them is not zero, `host.inspect()` says what: the scope trees, naming the
+module or operation that still holds an entry, plus the hosted services that
+never stopped and the operations that never settled.
+
+## Diagnostics
+
+The framework narrates its own lifecycle. Pass `onDiagnostic` and you get every
+transition as it happens: `application.starting` / `running` / `stopping` /
+`stopped` / `failed`, `application.preflight.error` and `.warning`,
+`module.binding` / `bound` / `failed` / `rollbackFailed`, `hostedService.*`,
+`operation.started` / `completed` / `cancelled` / `failed`, and the
+suppressions the notification and settings layers report.
+
+<!-- sample: docs/samples/diagnostics.ts -->
+
+```ts
+import { defineExtension } from '@kkdev92/vscode-ext-kit';
+import type { HostDiagnostic } from '@kkdev92/vscode-ext-kit';
+
+import { projectsModule } from './commands-and-services.js';
+
+/** The last few lifecycle events, for a "report an issue" command to attach. */
+const recent: HostDiagnostic[] = [];
+
+export const app = defineExtension({
+  name: 'Sample',
+  modules: [projectsModule],
+  // Called synchronously as the host starts, binds modules, runs operations and
+  // stops. Keep it cheap: it is not awaited, and an exception here is swallowed
+  // rather than allowed to affect the lifecycle it is watching.
+  onDiagnostic: (diagnostic) => {
+    recent.push(diagnostic);
+    if (recent.length > 100) {
+      recent.shift();
+    }
+  },
+});
+
+/**
+ * `application.shutdownTimeout` is the one worth reading first.
+ *
+ * It means the stop budget ran out and the remaining work was abandoned rather
+ * than awaited. `details` says which phase ran out, how long it waited, which
+ * hosted service was inside its `stop`, which operations never settled, and
+ * which resource scopes still held entries — ids and counts, never arguments
+ * or payloads.
+ */
+export function unfinishedAtShutdown(): readonly HostDiagnostic[] {
+  return recent.filter((diagnostic) => diagnostic.event === 'application.shutdownTimeout');
+}
+```
+
+The event name is a string and `details` is plain data, so this is a stream to
+log, count or attach to a bug report — not an event bus. Delivery is
+best-effort by design: a listener that throws is ignored, and nothing waits for
+one.
+
+`application.shutdownTimeout` deserves the special attention above because it is
+the one event that reports something the framework could not do. VS Code races
+every extension's deactivation against a few seconds and then exits; the
+framework's own budget sits inside that, and past it pending work is abandoned.
+Knowing _which_ work is the difference between a mystery and a fix.
+
 ## Keeping package.json honest
 
 VS Code reads the manifest before any extension code runs, so `src` and

docs/samples/diagnostics.ts

@@ -0,0 +1,34 @@
+import { defineExtension } from '@kkdev92/vscode-ext-kit';
+import type { HostDiagnostic } from '@kkdev92/vscode-ext-kit';
+
+import { projectsModule } from './commands-and-services.js';
+
+/** The last few lifecycle events, for a "report an issue" command to attach. */
+const recent: HostDiagnostic[] = [];
+
+export const app = defineExtension({
+  name: 'Sample',
+  modules: [projectsModule],
+  // Called synchronously as the host starts, binds modules, runs operations and
+  // stops. Keep it cheap: it is not awaited, and an exception here is swallowed
+  // rather than allowed to affect the lifecycle it is watching.
+  onDiagnostic: (diagnostic) => {
+    recent.push(diagnostic);
+    if (recent.length > 100) {
+      recent.shift();
+    }
+  },
+});
+
+/**
+ * `application.shutdownTimeout` is the one worth reading first.
+ *
+ * It means the stop budget ran out and the remaining work was abandoned rather
+ * than awaited. `details` says which phase ran out, how long it waited, which
+ * hosted service was inside its `stop`, which operations never settled, and
+ * which resource scopes still held entries — ids and counts, never arguments
+ * or payloads.
+ */
+export function unfinishedAtShutdown(): readonly HostDiagnostic[] {
+  return recent.filter((diagnostic) => diagnostic.event === 'application.shutdownTimeout');
+}

src/foundation/application/application.ts

@@ -29,7 +29,11 @@ import type {
 import { AsyncCallbackError, PreflightError, ScopeCleanupError } from '../internal/errors.js';
 import { claimRejection, isThenable } from '../internal/thenable.js';
 import { createApplicationHost } from '../hosting/application-host.js';
-import type { ApplicationHost, HostDiagnostic } from '../hosting/application-host.js';
+import type {
+  ApplicationHost,
+  HostDiagnostic,
+  HostInspection,
+} from '../hosting/application-host.js';
 import { StopReason } from '../hosting/host-state.js';
 import {
   CancellationReason,
@@ -177,6 +181,22 @@ export interface CreateApplicationOptions {
     | undefined;
 }
 
+/**
+ * What an Application still owns, as plain data.
+ *
+ * Extends the Host's view — scopes and state — with the work only the
+ * Application knows about: which hosted services are up, and which operations
+ * have not settled. Names and counts only.
+ */
+export interface ApplicationInspection extends HostInspection {
+  /** Hosted services that started and have not stopped, in start order. */
+  readonly hostedServices: readonly string[];
+  /** Ids of operations that started and have not settled. */
+  readonly operations: readonly string[];
+  /** Background hosted-service loops still being tracked. */
+  readonly backgroundTasks: number;
+}
+
 /**
  * A compiled plan wired to platform capabilities, ready for Extension Host
  * activation.
@@ -205,6 +225,11 @@ export interface Application {
    * rejects; cleanup failures are emitted as diagnostics.
    */
   deactivate(): Promise<void>;
+  /**
+   * What the framework still owns. Read by the Test Host's leak report, and by
+   * the Host itself when a shutdown runs out of budget.
+   */
+  inspect(): ApplicationInspection;
 }
 
 /**
@@ -234,11 +259,26 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
       ? createNoopLogger()
       : createLogger(options.logSink, { application: plan.name });
 
+  /** Hosted services that have started and not yet stopped, in start order. */
   const startedServices: {
     readonly definition: HostedServiceDefinition;
     readonly injected: Readonly<Record<string, unknown>>;
   }[] = [];
   const backgroundTasks: Promise<void>[] = [];
+  /**
+   * Operations that started and have not settled, keyed by id.
+   *
+   * Bookkeeping on a stream that already exists rather than a second one: the
+   * executor stamps every `operation.*` diagnostic with its id, so the events
+   * flowing through this file are enough to answer "what is still running?"
+   * when a shutdown runs out of budget.
+   */
+  const inFlightOperations = new Map<
+    string,
+    { readonly name: string; readonly kind: string; readonly startedAt: number }
+  >();
+  /** The hosted service currently inside its `stop`, if any. */
+  let stoppingService: string | undefined;
 
   // Observability must never interfere: a throwing observer cannot be allowed
   // to fail activation, an operation, or cleanup.
@@ -250,13 +290,56 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
     }
   };
 
+  /** Reads a diagnostic field that is `unknown` by contract. */
+  const text = (value: unknown): string => (typeof value === 'string' ? value : '');
+
   const emitOperationDiagnostic = (
     event: string,
     details: Readonly<Record<string, unknown>>
   ): void => {
+    const id: unknown = details['operationId'];
+    if (typeof id === 'string') {
+      if (event === 'operation.started') {
+        inFlightOperations.set(id, {
+          name: text(details['name']),
+          kind: text(details['kind']),
+          startedAt: Date.now(),
+        });
+      } else if (
+        event === 'operation.completed' ||
+        event === 'operation.cancelled' ||
+        event === 'operation.failed'
+      ) {
+        // The executor settles an operation with exactly one of these, before
+        // its `finally` reports any cleanup failure.
+        inFlightOperations.delete(id);
+      }
+    }
     emitDiagnostic({ event, details });
   };
 
+  /**
+   * Who is still holding the shutdown budget when it runs out.
+   *
+   * Ids, names and counts. Command arguments, webview payloads and secret
+   * values are deliberately absent: naming the owner is enough to act on, and
+   * a diagnostic that carried the work's own data would put it wherever the
+   * log goes.
+   */
+  const describeRemaining = (): Readonly<Record<string, unknown>> => ({
+    hostedServices: {
+      started: startedServices.map((started) => started.definition.id),
+      ...(stoppingService === undefined ? {} : { stopping: stoppingService }),
+    },
+    operations: [...inFlightOperations].map(([id, operation]) => ({
+      id,
+      name: operation.name,
+      kind: operation.kind,
+      elapsedMs: Date.now() - operation.startedAt,
+    })),
+    backgroundTasks: backgroundTasks.length,
+  });
+
   /**
    * Waits for tracked background loops to settle, never past the remaining
    * budget. The tasks already carry their own catch handlers, so abandoning an
@@ -277,7 +360,12 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
     });
     try {
       if ((await Promise.race([Promise.all(pending), timeout])) === 'timeout') {
-        emitDiagnostic({ event: 'application.shutdownTimeout', details: { phase: 'background' } });
+        emitDiagnostic({
+          event: 'application.shutdownTimeout',
+          // `pending` rather than the tracked list: this drain took ownership
+          // of those promises, so `describeRemaining` no longer counts them.
+          details: { phase: 'background', pending: pending.length, ...describeRemaining() },
+        });
       }
     } finally {
       if (timer !== undefined) {
@@ -340,6 +428,11 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
   ): Promise<void> => {
     for (let index = startedServices.length - 1; index >= 0; index -= 1) {
       const started = startedServices[index];
+      // Truncated as the loop goes, so `startedServices` always names what is
+      // still up rather than everything that ever started -- which is what a
+      // shutdown-timeout diagnostic has to report. Safe while iterating
+      // backwards: only entries at or after the current index are removed.
+      startedServices.length = index;
       const stop = started?.definition.stop;
       if (started === undefined || stop === undefined) {
         continue;
@@ -348,6 +441,7 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
       const logger = rootLogger.withFields({ hostedServiceId: definition.id });
       const context: HostedServiceStopContext = { signal, logger, remainingMs };
       emitDiagnostic({ event: 'hostedService.stopping', details: { id: definition.id } });
+      stoppingService = definition.id;
       try {
         await stop(context, injected);
         emitDiagnostic({ event: 'hostedService.stopped', details: { id: definition.id } });
@@ -357,14 +451,16 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
           event: 'hostedService.failed',
           details: { id: definition.id, error },
         });
+      } finally {
+        stoppingService = undefined;
       }
     }
-    startedServices.length = 0;
   };
 
   const host = createApplicationHost({
     name: plan.name,
     shutdownTimeoutMs: plan.shutdown.timeoutMs,
+    describeRemaining,
     ...(options.onDiagnostic === undefined ? {} : { onDiagnostic: options.onDiagnostic }),
 
     async start({ registrations, resources, signal }) {
@@ -1038,6 +1134,15 @@ export function createApplication(options: CreateApplicationOptions): Applicatio
     deactivate(): Promise<void> {
       return host.stop(StopReason.Deactivate);
     },
+
+    inspect(): ApplicationInspection {
+      return {
+        ...host.inspect(),
+        hostedServices: startedServices.map((started) => started.definition.id),
+        operations: [...inFlightOperations.keys()],
+        backgroundTasks: backgroundTasks.length,
+      };
+    },
   };
 }