Skip to content

Scoped uninstall destabilizes a sibling gateway sandbox #8663

Description

@apurvvkumaria

Contributor preflight (required)

Before investigating or changing code:

  1. Read WRITING.md in full.
  2. Read CONTRIBUTING.md in full.
  3. Run npm run dev:doctor from a clean checkout before creating a branch.
  4. Follow both guides for code, tests, issue updates, commit messages, DCO, commit verification, documentation review, and the pull request.

Problem

A gateway-scoped uninstall of an alternate gateway removes the selected gateway successfully but destabilizes a healthy sandbox owned by the default gateway. This violates the multi-gateway isolation contract.

Confirmed evidence

  • Failing run: E2E run 31337932239, exact head 1a8e6bd3198ec54c7682f98b23141bdaac716160.
  • The concurrent-gateway-ports job failed in both attempts: attempt 1 and attempt 2.
  • The attempt 2 evidence artifact contains the command and phase records.
  • The runner used Ubuntu 24.04 and OpenShell 0.0.99.
  • Sandbox A onboarded on gateway nemoclaw at port 8080 and reached Ready.
  • Sandbox B onboarded on gateway nemoclaw-18080 at port 18080 and reached Ready.
  • Before uninstall, both gateway listeners, dashboard forwards, sandboxes, and port-scoped registries passed their isolation assertions.
  • NEMOCLAW_GATEWAY_PORT=18080 nemoclaw uninstall --yes --destroy-user-data exited 0 and reported removing gateway B and its scoped state.
  • Immediately afterward, openshell sandbox list -g nemoclaw showed sandbox A transition from Provisioning to terminal Error.

The test failed with:

e2e-cgp-a reached terminal phase 'Error' on nemoclaw

Reproduction

Run the existing live scenario with Docker and the supported OpenShell CLI available:

NEMOCLAW_RUN_LIVE_E2E=1 npx vitest run --project e2e-live test/e2e/live/concurrent-gateway-ports.test.ts

The scenario onboards a default-port sandbox and a port-18080 sandbox, verifies both are isolated and healthy, then uninstalls only port 18080 and probes the default sandbox.

Likely implementation surface

  • src/lib/actions/uninstall/run-plan.ts
  • src/lib/onboard/host-gateway-process.ts
  • src/lib/onboard/gateway-process-target-identity.ts
  • src/lib/onboard/gateway-process-identity.ts
  • src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts
  • src/lib/actions/uninstall/run-plan-gateway-service.test.ts
  • src/lib/onboard/host-gateway-process-target.test.ts
  • src/lib/onboard/host-gateway-process.test.ts
  • test/e2e/live/concurrent-gateway-ports.test.ts

Likely cause — hypothesis, not confirmed

The scoped uninstall requests a stop using gateway B's state directory and expected gateway name and port. The resulting teardown may signal a process shared with or owned by the default gateway. A second possibility is an OpenShell 0.0.99 multi-gateway lifecycle coupling after gateway B stops.

Do not assume which branch is true. Capture both gateways' PID files, command lines, listener ownership, and gateway and sandbox state immediately before and after the selected PID stops. Base the fix on that ownership evidence.

Acceptance criteria

  • With healthy sandboxes on ports 8080 and 18080, uninstalling port 18080 removes only sandbox B, gateway B's registration and listener, and B-scoped state.
  • The uninstall never signals the default gateway's process or removes its runtime state.
  • Gateway A stays listening and sandbox A remains Ready or Running across multiple post-uninstall probes.
  • NEMOCLAW_GATEWAY_PORT=8080 nemoclaw list still succeeds and sandbox A's dashboard forward remains usable.
  • PID and command-line targeting fails closed when ownership cannot be proven.
  • Scoped teardown never falls back to a host-wide gateway sweep.
  • Full single-gateway uninstall still removes its owned host gateway process and resources.
  • Regression coverage proves that two live gateway processes cannot cross-match by PID file, command line, name, or port.
  • The existing concurrent-gateway-ports live E2E target passes in the authoritative CI environment.

Targeted verification

npx vitest run --project cli \
  src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts \
  src/lib/actions/uninstall/run-plan-gateway-service.test.ts \
  src/lib/onboard/host-gateway-process-target.test.ts \
  src/lib/onboard/host-gateway-process.test.ts

Then run the concurrent-gateway-ports live E2E target. If the user-visible teardown or recovery contract changes, update docs/manage-sandboxes/uninstall-nemoclaw.mdx and docs/reference/commands.mdx, then complete the required documentation-writer review.

Related work

None of these tracks the sibling-isolation failure reproduced here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area: e2eEnd-to-end tests, nightly failures, or validation infrastructurearea: installInstall, setup, prerequisites, or uninstall flowarea: sandboxOpenShell sandbox lifecycle, runtime, config, or recovery

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions