Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/manage-sandboxes/uninstall-nemoclaw.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,13 @@ Rerun `NEMOCLAW_GATEWAY_PORT=<port> $$nemoclaw uninstall` with the gateway port
For an externally supervised authority, uninstall preserves the local gateway state used by the running process in both full and gateway-scoped cleanup.
It also preserves the gateway process, supervisor resources, marked Linux unit, Docker resources, OpenShell binaries, and the declared external state directory.
A custom-port uninstall does not stop or remove the default gateway service or its environment file.
Uninstall does not stop an `openshell-gateway` process that another non-root user owns and that this installation did not record.
It names the owner and process ID, leaves that process running, and continues with the remaining cleanup.
If no other cleanup fails, uninstall exits with status `0` even though that process can keep its port in use.
Uninstall still tries to stop a `root`-owned process and the gateway process that this installation recorded.
If either of those stops fails, uninstall prints `sudo kill -9 <pid>` for the process.
A gateway-scoped uninstall and every `--all-gateway-ports` pass exit nonzero after that failure.
A single full uninstall reports the process and continues.
Comment thread
apurvvkumaria marked this conversation as resolved.
Before scoped cleanup stops a Docker gateway process, including a managed default gateway service, NemoClaw requires two Docker namespace proofs.
The selected Docker gateway configuration and any running gateway process must use the state-root-specific OpenShell sandbox namespace that NemoClaw generated.
Because the supported OpenShell Podman schema does not expose `sandbox_namespace`, scoped Podman uninstall fails closed before signaling and preserves the gateway runtime evidence and local state.
Expand Down
7 changes: 7 additions & 0 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4151,6 +4151,13 @@ Rerun `NEMOCLAW_GATEWAY_PORT=<port> $$nemoclaw uninstall` with the gateway port
For an externally supervised authority, uninstall preserves the selected local gateway state in both full and gateway-scoped cleanup.
It also preserves the gateway process, supervisor resources, marked Linux unit, Docker resources, OpenShell binaries, and the declared external state directory.
A custom-port uninstall does not stop or remove the default gateway service or its environment file.
Uninstall does not stop an `openshell-gateway` process that another non-root user owns and that this installation did not record.
It names the owner and process ID, leaves that process running, and continues with the remaining cleanup.
If no other cleanup fails, uninstall exits with status `0` even though that process can keep its port in use.
Uninstall still tries to stop a `root`-owned process and the gateway process that this installation recorded.
If either of those stops fails, uninstall prints `sudo kill -9 <pid>` for the process.
A gateway-scoped uninstall and every `--all-gateway-ports` pass exit nonzero after that failure.
A single full uninstall reports the process and continues.
Before scoped cleanup stops a Docker gateway process, including a managed default gateway service, NemoClaw requires two Docker namespace proofs.
The selected Docker gateway configuration and any running gateway process must use the state-root-specific OpenShell sandbox namespace that NemoClaw generated.
Because the supported OpenShell Podman schema does not expose `sandbox_namespace`, scoped Podman uninstall fails closed before signaling and preserves the gateway runtime evidence and local state.
Expand Down
3 changes: 3 additions & 0 deletions docs/reference/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,9 @@ openshell gateway list

If the gateway name and its port-scoped state remain, treat it as a second environment and select that port for cleanup.
If the gateway is absent but the port still listens, cleanup did not stop the listener; follow the process or service remediation printed by uninstall before you retry.
If uninstall reported that it kept an `openshell-gateway` process owned by another user running, that process still holds the port.
This can happen after uninstall exits successfully because NemoClaw does not treat another user's process as a cleanup failure.
Ask that user to stop the process, or onboard under a different `NEMOCLAW_GATEWAY_PORT`.

Remove one environment by selecting its port:

Expand Down
99 changes: 99 additions & 0 deletions src/lib/actions/uninstall/run-plan-foreign-user-gateway.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import fs from "node:fs";
import os from "node:os";
import path from "node:path";

import { describe, expect, it, vi } from "vitest";

import { type RunResult, runUninstallPlan } from "./run-plan";

const HOST_GATEWAY_PID = 9999043;

function ok(stdout = ""): RunResult {
return { status: 0, stdout, stderr: "" };
}

function notFound(): RunResult {
return { status: 1, stdout: "", stderr: "" };
}

function uninstallWithHostGatewayOwnedBy(uid: number): {
errors: string[];
exitCode: number;
} {
const tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-uninstall-foreign-"));
const errors: string[] = [];
const psResults = new Map<string, RunResult>([
["stat=", ok("S\n")],
["args=", ok("/usr/local/bin/openshell-gateway\n")],
["user=", ok("otheruser\n")],
["uid=", ok(`${uid}\n`)],
]);
const run = (command: string, args: string[]): RunResult =>
command === "pgrep"
? args.some((arg) => arg.includes("openshell-gateway"))
? ok(`${HOST_GATEWAY_PID}\n`)
: notFound()
: command === "ps"
? (psResults.get(args.at(-1) ?? "") ?? notFound())
: command === "openshell" && args.join(" ") === "gateway list -o json"
? ok("[]")
: ok();
try {
const result = runUninstallPlan(
{ assumeYes: true, deleteModels: false, keepOpenShell: false },
{
commandExists: (command) => command === "pgrep" || command === "openshell",
env: { HOME: tmpHome, NO_COLOR: "1" },
error: (message) => errors.push(message),
existsSync: () => false,
isTty: false,
kill: () => false,
log: vi.fn(),
requireCompleteGatewayProcessCleanup: true,
resolveGatewayTeardownAuthority: ({ gatewayName, gatewayPort }) => ({
gatewayName,
gatewayPort,
mode: "nemoclaw-managed",
source: "standalone",
endpoint: null,
stateDir: null,
supervisor: null,
requiredCapabilities: [],
}),
rmSync: vi.fn(),
run,
runDocker: () => ok(),
},
);
return { errors, exitCode: result.exitCode };
} finally {
fs.rmSync(tmpHome, { force: true, recursive: true });
}
}

describe("uninstall with a host gateway owned by another user", () => {
it("completes when the only unstoppable gateway process belongs to another user", () => {
const { errors, exitCode } = uninstallWithHostGatewayOwnedBy((process.getuid?.() ?? 0) + 1);

expect(exitCode).toBe(0);
expect(errors).toContainEqual(
`Kept otheruser-owned host openshell-gateway process ${HOST_GATEWAY_PID} running. ` +
"Cleanup does not stop a gateway process that another user owns.",
);
expect(errors).not.toContainEqual(
"Cannot continue uninstall because host gateway process cleanup did not complete.",
);
});

it("still fails when the current user's own gateway process cannot be stopped", () => {
const { errors, exitCode } = uninstallWithHostGatewayOwnedBy(process.getuid?.() ?? 0);

expect(exitCode).toBe(1);
expect(errors).toContainEqual(
"Cannot continue uninstall because host gateway process cleanup did not complete.",
);
});
});
97 changes: 96 additions & 1 deletion src/lib/onboard/host-gateway-process.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ function psResponses(
cmdline?: string;
exited: Set<number>;
owner?: string;
uid?: number;
},
): [string, RunResult | ((args: string[]) => RunResult)][] {
return [
Expand All @@ -65,9 +66,16 @@ function psResponses(
`ps -p ${pid} -o args=`,
ok(opts.cmdline ?? `/home/test/.local/bin/openshell-gateway --port 8080\n`),
],
...(opts.uid === undefined
? []
: [[`ps -p ${pid} -o uid=`, ok(`${opts.uid}\n`)] as [string, RunResult]]),
];
}

function otherUserUid(): number {
return (process.getuid?.() ?? 0) + 1;
}

describe("host gateway cleanup boundaries", () => {
it.each([
["free", 0, true],
Expand Down Expand Up @@ -350,7 +358,7 @@ describe("stopHostGatewayProcesses", () => {
it("prints sudo remediation when a privileged host gateway cannot be killed", () => {
const responses = new Map<string, RunResult | ((args: string[]) => RunResult)>([
[PGREP_KEY, ok("9999042\n")],
...psResponses(9999042, { exited: new Set(), owner: "root" }),
...psResponses(9999042, { exited: new Set(), owner: "root", uid: 0 }),
]);
const { run } = makeRun(responses);
const warn = vi.fn();
Expand All @@ -377,6 +385,93 @@ describe("stopHostGatewayProcesses", () => {
);
});

it("leaves a swept host gateway owned by another user running without failing cleanup", () => {
const responses = new Map<string, RunResult | ((args: string[]) => RunResult)>([
[PGREP_KEY, ok("9999043\n")],
...psResponses(9999043, { exited: new Set(), owner: "otheruser", uid: otherUserUid() }),
]);
const { run } = makeRun(responses);
const kill = vi.fn(() => false);
const warn = vi.fn();

const result = stopHostGatewayProcesses(
{
run,
kill,
env: { USER: "tester" },
commandExists: () => true,
warn,
},
{
killWaitMs: 0,
stateDir: fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-host-gateway-")),
termWaitMs: 0,
},
);

expect(result.foreignUserPids).toEqual([9999043]);
expect(result.failed).toEqual([]);
expect(result.sudoRemediationPids).toEqual([]);
expect(result.stopped).toEqual([]);
expect(kill).not.toHaveBeenCalled();
expect(warn).toHaveBeenCalledWith(
"Kept otheruser-owned host openshell-gateway process 9999043 running. " +
"Cleanup does not stop a gateway process that another user owns.",
);
});
Comment thread
apurvvkumaria marked this conversation as resolved.

it("stops a foreign-user gateway recorded by this installation", () => {
const pid = 9999045;
const stateDir = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-host-gateway-"));
fs.writeFileSync(path.join(stateDir, "openshell-gateway.pid"), `${pid}\n`);
const exited = new Set<number>();
const responses = new Map<string, RunResult | ((args: string[]) => RunResult)>([
[PGREP_KEY, notFound()],
...psResponses(pid, { exited, owner: "otheruser", uid: otherUserUid() }),
]);
const { run } = makeRun(responses);
const kill = vi.fn<HostGatewayProcessDeps["kill"]>((targetPid, signal) => {
signal === "SIGTERM" && exited.add(targetPid);
return true;
});

const result = stopHostGatewayProcesses(
{ run, kill, env: { USER: "tester" }, commandExists: () => true, log: vi.fn() },
{ stateDir },
);

expect(result.stopped).toEqual([pid]);
expect(result.foreignUserPids).toEqual([]);
expect(kill).toHaveBeenCalledWith(pid, "SIGTERM");
});

it("still stops a swept host gateway owned by the current user", () => {
const exited = new Set<number>();
const responses = new Map<string, RunResult | ((args: string[]) => RunResult)>([
[PGREP_KEY, ok("9999044\n")],
...psResponses(9999044, { exited, uid: process.getuid?.() ?? 0 }),
]);
const { run } = makeRun(responses);
const kill = vi.fn<HostGatewayProcessDeps["kill"]>((pid, signal) => {
signal === "SIGTERM" && exited.add(pid);
return true;
});

const result = stopHostGatewayProcesses(
{
run,
kill,
env: { USER: "tester" },
commandExists: () => true,
log: vi.fn(),
},
{ stateDir: fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-host-gateway-")) },
);

expect(result.stopped).toEqual([9999044]);
expect(result.foreignUserPids).toEqual([]);
});

it("skips pgrep sweep when explicit PIDs are passed (drift restart)", () => {
// Use a PID above the Linux kernel pid_max default (4194304) so that the
// production code's `/proc/<pid>/cmdline` probe always misses and the
Expand Down
36 changes: 36 additions & 0 deletions src/lib/onboard/host-gateway-process.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ export interface StopHostGatewayOptions {

export interface StopHostGatewayResult {
failed: number[];
foreignUserPids?: number[];
/** Whether a requested pgrep fallback completed with a usable result. */
orphanScanComplete?: boolean;
ownershipFailures?: string[];
Expand Down Expand Up @@ -204,6 +205,31 @@ function pidOwner(pid: number, deps: HostGatewayProcessDeps): string | null {
return result.stdout.trim() || null;
}

function pidOwnerUid(pid: number, deps: HostGatewayProcessDeps): number | null {
const result = deps.run("ps", ["-p", String(pid), "-o", "uid="], { env: deps.env });
if (result.status !== 0) return null;
const uid = Number.parseInt(result.stdout.trim(), 10);
return Number.isInteger(uid) ? uid : null;
}

function pidBelongsToAnotherUser(pid: number, deps: HostGatewayProcessDeps): boolean {
const currentUid = typeof process.getuid === "function" ? process.getuid() : -1;
if (currentUid < 0) return false;
const uid = pidOwnerUid(pid, deps);
if (uid === null || uid === 0) return false;
return uid !== currentUid;
}

function warnForeignUserGateway(pid: number, deps: HostGatewayProcessDeps): void {
const warn = deps.warn ?? ((message: string) => console.warn(message));
const owner = pidOwner(pid, deps);
const ownerLabel = owner ? `${owner}-owned` : "another user's";
warn(
`Kept ${ownerLabel} host openshell-gateway process ${pid} running. ` +
"Cleanup does not stop a gateway process that another user owns.",
);
}

function readOwnedRuntimeFile(filePath: string, uid: number): string | null {
if (typeof fs.constants.O_NOFOLLOW !== "number") return null;
let descriptor: number | undefined;
Expand Down Expand Up @@ -417,6 +443,7 @@ export function stopHostGatewayProcesses(
const candidates = new Map<number, Set<string>>();
const result: StopHostGatewayResult = {
failed: [],
foreignUserPids: [],
orphanScanComplete: true,
ownershipFailures: [],
skippedDeadPids: [],
Expand Down Expand Up @@ -546,6 +573,15 @@ export function stopHostGatewayProcesses(
}
continue;
}
if (
!options.scopedGatewayStop &&
!sources.has("pid-file") &&
pidBelongsToAnotherUser(pid, deps)
) {
(result.foreignUserPids ??= []).push(pid);
warnForeignUserGateway(pid, deps);
continue;
}

const stopResult = tryStopPid(
pid,
Expand Down
Loading