From 6bf1f84e503c7c812d3378a10f133e82c16fbcad Mon Sep 17 00:00:00 2001 From: "Mariusz (via Fable)" Date: Mon, 24 Aug 2026 11:25:18 +0000 Subject: [PATCH 1/4] =?UTF-8?q?propose:=20=C2=A712b=20m2o=20desktop=20prov?= =?UTF-8?q?isioning=20+=20launch-desktop.sh=20wrapper?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Captures the m2 / m2.2 host topology, path & DNS gotchas, first-boot `unhealthy` window, expected WARNs on m2.2, and the socat-relay pattern for wiring m2.2-hosted desktops into m2's Guacamole (guacamole-full). Adds `scripts/launch-m2o-desktop.sh`, an idempotent wrapper that runs provision.sh, waits for healthy, creates the guacd relay (m2.2 only), upserts the Guacamole RDP row on m2, and grants perms — one command end-to-end. Installed on both hosts as `launch-desktop.sh`. Amendment trigger: pattern that had to be reconstructed from scratch during euroclean-m2o provisioning (2026-08-24) — belongs in the playbook. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 9 ++ PLAYBOOK.md | 125 +++++++++++++++++ scripts/launch-m2o-desktop.sh | 248 ++++++++++++++++++++++++++++++++++ 3 files changed, 382 insertions(+) create mode 100755 scripts/launch-m2o-desktop.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index f4d269f..c09d32d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +## [Unreleased] — 2026-08-24 +### Added +- Section 12b: m2o Desktop Provisioning (RDP + cross-host Guacamole) — captures the m2/m2.2 topology, path/DNS gotchas, first-boot `unhealthy` wait, expected WARNs on m2.2, and the socat-relay pattern for wiring m2.2 desktops into m2's Guacamole. +- `scripts/launch-m2o-desktop.sh` — idempotent one-command wrapper around `provision.sh` + relay + Guacamole upsert + perm grant. Installed on m2 (`~/m2o/desktop/launch-desktop.sh`) and m2.2 (`~/machinemachine-core/m2o/desktop/launch-desktop.sh`). + +### Authors +- Mariusz (operator) — hit the pain, asked for the doc + script +- Fable — end-to-end euroclean provisioning + doc/script extraction + ## [0.2.0] — 2026-02-19 ### Added - Section 10: Orchestrator Pattern — conductor/player separation, structured spawn handoff, when to break the rule diff --git a/PLAYBOOK.md b/PLAYBOOK.md index e073a66..7736afe 100644 --- a/PLAYBOOK.md +++ b/PLAYBOOK.md @@ -617,6 +617,131 @@ Each cycle, the fleet gets more capable. This is not metaphor — it is the mech --- +## 12b. m2o Desktop Provisioning (RDP + cross-host Guacamole) + +Every fleet agent runs on an **m2o desktop** — a persistent Ubuntu container with `x11vnc`, `xrdp`, `guacd`, `ttyd`, and Hermes baked in (primus image). Two hosts run desktops today: + +| Host | LAN IP(s) | SSH user | provision.sh path | Local Guacamole? | +|------|-----------|----------|-------------------|------------------| +| **m2** (primary) | `192.168.31.224`, `192.168.31.28` | `m2` | `~/m2o/desktop/provision.sh` | ✅ `guacamole-full` + `guacamole-db` (Coolify service `e0o8o8cowkswcwsgs4so48s8`) | +| **m2.2** (secondary) | `192.168.31.34` | `m2.2` | `~/machinemachine-core/m2o/desktop/provision.sh` | ❌ own Coolify, no Guacamole stack | + +> **Path gotcha:** on m2.2 the script is under `~/machinemachine-core/m2o/`, not `~/m2o/`. The `m2o-provision` skill doc uses the upstream `~/m2o/` path — treat as advisory, not literal for m2.2. + +> **DNS gotcha:** from inside fleet docker containers (e.g. anything on the `coolify` network), both `m2` and `m2.2` frequently resolve to the *same* tailnet address (whichever host the container runs on). When you need to reach the *other* host, use its LAN IP, not the name. + +### Spawn a desktop — one command, full setup + +Use the **`launch-desktop.sh`** wrapper (installed at `~/m2o/desktop/launch-desktop.sh` on m2 and `~/machinemachine-core/m2o/desktop/launch-desktop.sh` on m2.2; canonical source: `scripts/launch-m2o-desktop.sh` in this repo). It runs `provision.sh`, waits for healthy, creates the socat guacd relay (m2.2 only), upserts the Guacamole RDP row on m2, and grants perms — all idempotent. + +```bash +# on the target host, as its own user (m2 or m2.2) +cd ~/m2o/desktop # or ~/machinemachine-core/m2o/desktop on m2.2 +./launch-desktop.sh [--vnc-pass X] [--grant-users guacadmin,m2,...] +``` + +Flags: `--relay-port ` (default: auto-pick next `X4822`), `--m2-ssh ` (default `m2@192.168.31.224`), `--skip-guacamole`, `--force-recreate`. + +**Raw provision (if you only want the container, no Guacamole wiring):** + +```bash +cd ~/m2o/desktop # or ~/machinemachine-core/m2o/desktop on m2.2 +./provision.sh [vnc_password] # e.g. ./provision.sh euroclean +``` + +Creates container `-m2o` on the `coolify` docker network + named volumes `-{agent-home,workspace,m2home}` (survive re-provision). `M2_GPT_API_KEY` is injected from `~/.m2-gpt-key` (host side, per-user). Fleet standard 2026-07-09: **RDP is the default access path**; VNC is fallback. + +### First-boot wait (~2 min "unhealthy" is normal) + +The entrypoint does a long `rm -rf /home` before x11vnc/xrdp start. The container will show `unhealthy` and ports 5900/3389 will refuse connections for 60–120s. Don't kill it — the health check flips once services come up. To watch: + +```bash +docker inspect -m2o --format '{{.State.Health.Status}}' +docker exec -m2o bash -lc 'ps -eo pid,etime,cmd | grep -E "rm -rf|x11vnc|xrdp|guacd" | grep -v grep' +``` + +### Expected WARNs on m2.2 (harmless — not bugs) + +Because m2.2 doesn't yet run its own Guacamole or console-auth: + +- `WARN: guacamole-db container not found — create the connection manually` +- `WARN console: console-auth:latest image missing — cred saved to secrets.env` + +Traefik route for the ttyd web console is written, but `/console/` won't work until console-auth is deployed to m2.2. + +### Wiring an m2.2 desktop into m2's Guacamole + +m2's `guacamole-full` needs to speak the guacd protocol to the desktop container (port 4822). On m2.2 that port lives inside the container on the `coolify` docker network — not published to the host. The established pattern is a **per-desktop `alpine/socat` relay** on m2.2 that publishes a unique host port and forwards to the desktop's internal `guacd:4822`. + +Existing port allocations (append your own): + +| Desktop | Host port on m2.2 | +|---------|-------------------| +| dealflow-legacy | 14822 | +| dealflow | 24823 | +| m2o-operator | 24822 | +| euroclean | 34822 | + +**Step 1 — publish guacd on m2.2:** + +```bash +# on m2.2, pick an unused port +docker run -d --name -guacd-relay --restart unless-stopped \ + --network coolify -p : \ + alpine/socat tcp-listen:,fork,reuseaddr tcp:-m2o:4822 +``` + +**Step 2 — insert Guacamole connection on m2** (via `guacamole-db`, user `root` / `guacamole_root_pass`): + +```sql +INSERT INTO guacamole_connection (connection_name, protocol, proxy_hostname, proxy_port) +VALUES (' Desktop', 'rdp', '192.168.31.34', ); +SET @id = LAST_INSERT_ID(); + +INSERT INTO guacamole_connection_parameter (connection_id, parameter_name, parameter_value) VALUES + (@id,'hostname','127.0.0.1'), (@id,'port','3389'), + (@id,'username','developer'), (@id,'password',''), + (@id,'width','1920'), (@id,'height','1080'), (@id,'color-depth','32'), + (@id,'security','any'), (@id,'ignore-cert','true'), + (@id,'enable-drive','true'), (@id,'drive-name','Shared'), + (@id,'drive-path','/home/developer/Desktop/Shared'), + (@id,'create-drive-path','true'), (@id,'enable-sftp','false'); + +INSERT INTO guacamole_connection_permission (entity_id, connection_id, permission) +SELECT e.entity_id, @id, p.perm +FROM guacamole_entity e +CROSS JOIN (SELECT 'READ' AS perm UNION ALL SELECT 'UPDATE' + UNION ALL SELECT 'DELETE' UNION ALL SELECT 'ADMINISTER') p +WHERE e.name = 'guacadmin' AND e.type = 'USER'; +``` + +**Critical schema gotchas** (inherited from m2's Guacamole): + +- `proxy_hostname` / `proxy_port` live on the `guacamole_connection` table, **not** on `guacamole_connection_parameter`. +- `hostname` inside the connection parameters stays `127.0.0.1` — guacd runs *inside* the desktop container, right next to x11vnc/xrdp. +- Use the `root` mysql user for writes. `guacamole_user` is SELECT-only. + +**Naming standard 2026-07-09:** plain `' Desktop'` = RDP (default). Append `' (VNC)'` only for the fallback VNC row if you also create one. + +### Access without Guacamole (m2.2 direct) + +```bash +IP=$(docker inspect -m2o --format '{{(index .NetworkSettings.Networks "coolify").IPAddress}}') +# from m2.2: RDP client → $IP:3389 (developer / ) +# from a workstation: ssh -L 3389:$IP:3389 m2.2@192.168.31.34, then RDP to localhost:3389 +``` + +### Managing + +```bash +docker ps --filter name=-m2o # list all m2o desktops on this host +docker rm -f -m2o && ./provision.sh # rebuild (volumes preserved) +docker volume ls | grep - # named volumes +docker inspect -m2o --format '{{.State.Health.Status}}' +``` + +--- + ## 13. Infrastructure Architecture The full infrastructure spec lives in: **[sections/architecture.md](sections/architecture.md)** diff --git a/scripts/launch-m2o-desktop.sh b/scripts/launch-m2o-desktop.sh new file mode 100755 index 0000000..09c8aa7 --- /dev/null +++ b/scripts/launch-m2o-desktop.sh @@ -0,0 +1,248 @@ +#!/usr/bin/env bash +# launch-m2o-desktop.sh — spawn an m2o desktop and wire it into fleet Guacamole. +# +# Runs on either m2 (primary) or m2.2 (secondary) and does end-to-end setup: +# 1. provision the desktop container (if not present) via the host's provision.sh +# 2. wait for it to become healthy (tolerating the ~2 min first-boot rm -rf /home) +# 3. on m2.2 only: create a per-desktop alpine/socat guacd relay on m2.2's host +# 4. upsert an RDP row in m2's guacamole-db pointing at the desktop's guacd +# 5. grant Guacamole perms to a configurable user list +# +# Idempotent: safe to re-run. Existing container/relay are reused; the connection +# row is updated in place (matched by connection_name + protocol). +# +# Usage: launch-m2o-desktop.sh [OPTIONS] + +set -euo pipefail + +# ---------- defaults ---------- +VNC_PASS="agentdesktop" +RELAY_PORT="" # auto-pick if unset (X4822 scheme) +GRANT_USERS="guacadmin,m2" # comma-separated Guacamole USER names +M2_SSH="m2@192.168.31.224" # SSH target for m2 (fallback: 192.168.31.28) +M2_LAN_IP="192.168.31.34" # m2.2's LAN IP — used as proxy_hostname +GUAC_DB_ROOT_PASS="guacamole_root_pass" +GUAC_DB_NAME="guacamole_db" +SKIP_GUAC=false +FORCE_RECREATE=false + +usage() { + cat < [OPTIONS] + +Spawns m2o desktop '-m2o' on the current host and wires it into m2's Guacamole. + +Options: + --vnc-pass VNC/RDP password (default: agentdesktop) + --relay-port Host port on m2.2 for guacd relay (default: auto-pick) + --grant-users u1,u2,... Guacamole users to grant perms (default: guacadmin,m2) + --m2-ssh SSH target for m2 (default: m2@192.168.31.224) + --skip-guacamole Skip Guacamole wiring (provision + relay only) + --force-recreate Rebuild container even if it exists (volumes preserved) + -h, --help This help +EOF +} + +# ---------- parse args ---------- +[[ $# -lt 1 ]] && { usage; exit 1; } +case "$1" in -h|--help) usage; exit 0;; esac +NAME="$1"; shift +[[ "$NAME" =~ ^[a-z][a-z0-9-]*$ ]] || { echo "ERROR: name must be lowercase alnum/hyphen"; exit 1; } +while [[ $# -gt 0 ]]; do + case "$1" in + --vnc-pass) VNC_PASS="$2"; shift 2;; + --relay-port) RELAY_PORT="$2"; shift 2;; + --grant-users) GRANT_USERS="$2"; shift 2;; + --m2-ssh) M2_SSH="$2"; shift 2;; + --skip-guacamole) SKIP_GUAC=true; shift;; + --force-recreate) FORCE_RECREATE=true; shift;; + -h|--help) usage; exit 0;; + *) echo "ERROR: unknown option: $1"; usage; exit 1;; + esac +done + +CONTAINER="${NAME}-m2o" +RELAY_NAME="${NAME}-guacd-relay" +CONN_NAME="$(tr '[:lower:]' '[:upper:]' <<<"${NAME:0:1}")${NAME:1} Desktop" + +# ---------- detect host role ---------- +H=$(hostname) +case "$H" in + m2|m2-main|m2main) + HOST_ROLE="m2" + PROVISION_DIR="$HOME/m2o/desktop" + ;; + m2.2) + HOST_ROLE="m2.2" + PROVISION_DIR="$HOME/machinemachine-core/m2o/desktop" + ;; + *) + echo "ERROR: unrecognized hostname '$H' — must be m2 or m2.2"; exit 1;; +esac +echo ">> host role: $HOST_ROLE (hostname=$H)" +[[ -x "$PROVISION_DIR/provision.sh" ]] || { + echo "ERROR: provision.sh not found at $PROVISION_DIR/provision.sh"; exit 1; } + +# ---------- helper: run SQL on m2's guacamole-db ---------- +guac_sql() { + local sql="$1" + local cmd='DB=$(docker ps --format "{{.Names}}" | grep -m1 guacamole-db); ' + cmd+='exec docker exec -i "$DB" mysql -uroot -p'"$GUAC_DB_ROOT_PASS"' -Nse "$(cat)" '"$GUAC_DB_NAME" + if [[ "$HOST_ROLE" == "m2" ]]; then + printf '%s' "$sql" | bash -c "$cmd" + else + printf '%s' "$sql" | ssh -o BatchMode=yes "$M2_SSH" bash -c "'$cmd'" + fi +} + +# ---------- 1. provision ---------- +echo ">> [1/5] provision: $CONTAINER" +if docker inspect "$CONTAINER" >/dev/null 2>&1; then + if $FORCE_RECREATE; then + echo " --force-recreate: removing $CONTAINER (named volumes preserved)" + docker rm -f "$CONTAINER" >/dev/null + (cd "$PROVISION_DIR" && ./provision.sh "$NAME" "$VNC_PASS") + else + echo " already exists — skipping provision (use --force-recreate to rebuild)" + fi +else + (cd "$PROVISION_DIR" && ./provision.sh "$NAME" "$VNC_PASS") +fi + +# ---------- 2. wait for healthy + RDP ready ---------- +echo ">> [2/5] waiting for $CONTAINER — first-boot cleanup takes ~2 min" +DESKTOP_IP="" +for i in $(seq 1 60); do + status=$(docker inspect "$CONTAINER" --format '{{.State.Health.Status}}' 2>/dev/null || echo "?") + ip=$(docker inspect "$CONTAINER" \ + --format '{{(index .NetworkSettings.Networks "coolify").IPAddress}}' 2>/dev/null || true) + if [[ "$status" == "healthy" && -n "$ip" ]] && timeout 2 bash -c "echo > /dev/tcp/$ip/3389" 2>/dev/null; then + DESKTOP_IP="$ip" + echo " healthy — IP=$DESKTOP_IP, RDP:3389 open" + break + fi + printf ' [%3ds] status=%-10s ip=%-12s\n' $((i*5)) "$status" "${ip:--}" + sleep 5 +done +[[ -n "$DESKTOP_IP" ]] || { echo "ERROR: $CONTAINER never became healthy/ready"; exit 1; } + +# ---------- 3. guacd relay (m2.2 only) ---------- +GUAC_PROXY_HOST="" +GUAC_PROXY_PORT="" +if [[ "$HOST_ROLE" == "m2.2" ]]; then + echo ">> [3/5] guacd relay: $RELAY_NAME on m2.2" + if docker inspect "$RELAY_NAME" >/dev/null 2>&1; then + existing=$(docker inspect "$RELAY_NAME" \ + --format '{{range $p, $bs := .NetworkSettings.Ports}}{{range $bs}}{{.HostPort}} {{end}}{{end}}' \ + | awk '{print $1}') + echo " $RELAY_NAME already exists on host port ${existing:-?} — reusing" + GUAC_PROXY_PORT="${existing:-$RELAY_PORT}" + else + if [[ -z "$RELAY_PORT" ]]; then + # auto-pick next free X4822 (14822, 24822, 34822, ...) above the current max + max=$(docker ps -a --format '{{.Ports}}' \ + | grep -oE '0\.0\.0\.0:[0-9]+4822' \ + | awk -F: '{print $2}' | sort -n | tail -1) + if [[ -z "$max" ]]; then next=14822 + else next=$((max + 10000)) + fi + RELAY_PORT=$next + echo " auto-picked relay port: $RELAY_PORT" + fi + docker run -d --name "$RELAY_NAME" --restart unless-stopped \ + --network coolify -p "$RELAY_PORT:$RELAY_PORT" \ + alpine/socat "tcp-listen:$RELAY_PORT,fork,reuseaddr" "tcp:$CONTAINER:4822" >/dev/null + echo " created $RELAY_NAME → tcp-listen:$RELAY_PORT → $CONTAINER:4822" + GUAC_PROXY_PORT="$RELAY_PORT" + fi + GUAC_PROXY_HOST="$M2_LAN_IP" +else + echo ">> [3/5] host=m2 — no relay needed (guacamole-full reaches $CONTAINER:4822 on the coolify network)" + GUAC_PROXY_HOST="$CONTAINER" + GUAC_PROXY_PORT="4822" +fi + +# ---------- 4. Guacamole connection (upsert) ---------- +if $SKIP_GUAC; then + echo ">> [4/5] --skip-guacamole — skipping Guacamole wiring" +else + echo ">> [4/5] Guacamole: upsert '$CONN_NAME' → $GUAC_PROXY_HOST:$GUAC_PROXY_PORT" + ESC_NAME="${CONN_NAME//\'/\\\'}" + + existing_id=$(guac_sql "SELECT connection_id FROM guacamole_connection WHERE connection_name='$ESC_NAME' AND protocol='rdp' LIMIT 1;") + existing_id=$(echo "$existing_id" | tr -d '[:space:]') + + if [[ -n "$existing_id" ]]; then + echo " updating existing connection_id=$existing_id" + guac_sql " + UPDATE guacamole_connection + SET proxy_hostname='$GUAC_PROXY_HOST', proxy_port=$GUAC_PROXY_PORT + WHERE connection_id=$existing_id; + DELETE FROM guacamole_connection_parameter WHERE connection_id=$existing_id; + " + CONN_ID="$existing_id" + else + guac_sql " + INSERT INTO guacamole_connection (connection_name, protocol, proxy_hostname, proxy_port) + VALUES ('$ESC_NAME', 'rdp', '$GUAC_PROXY_HOST', $GUAC_PROXY_PORT); + " + CONN_ID=$(guac_sql "SELECT connection_id FROM guacamole_connection WHERE connection_name='$ESC_NAME' AND protocol='rdp' LIMIT 1;" | tr -d '[:space:]') + echo " inserted new connection_id=$CONN_ID" + fi + + guac_sql " + INSERT INTO guacamole_connection_parameter (connection_id, parameter_name, parameter_value) VALUES + ($CONN_ID,'hostname','127.0.0.1'), ($CONN_ID,'port','3389'), + ($CONN_ID,'username','developer'), ($CONN_ID,'password','$VNC_PASS'), + ($CONN_ID,'width','1920'), ($CONN_ID,'height','1080'), ($CONN_ID,'color-depth','32'), + ($CONN_ID,'security','any'), ($CONN_ID,'ignore-cert','true'), + ($CONN_ID,'enable-drive','true'), ($CONN_ID,'drive-name','Shared'), + ($CONN_ID,'drive-path','/home/developer/Desktop/Shared'), + ($CONN_ID,'create-drive-path','true'), ($CONN_ID,'enable-sftp','false'); + " + + # ---------- 5. perms ---------- + echo ">> [5/5] granting perms to: $GRANT_USERS" + IFS=',' read -ra USERS <<<"$GRANT_USERS" + for u in "${USERS[@]}"; do + u_trim=$(echo "$u" | tr -d '[:space:]') + [[ -z "$u_trim" ]] && continue + guac_sql " + INSERT IGNORE INTO guacamole_connection_permission (entity_id, connection_id, permission) + SELECT e.entity_id, $CONN_ID, p.perm + FROM guacamole_entity e + CROSS JOIN (SELECT 'READ' AS perm UNION ALL SELECT 'UPDATE' + UNION ALL SELECT 'DELETE' UNION ALL SELECT 'ADMINISTER') p + WHERE e.name='$u_trim' AND e.type='USER'; + " + echo " granted: $u_trim" + done + + # refresh guacamole-full so it picks up the new/updated row immediately + echo " refreshing guacamole-full (restart)" + if [[ "$HOST_ROLE" == "m2" ]]; then + FULL=$(docker ps --format '{{.Names}}' | grep -m1 guacamole-full || true) + [[ -n "$FULL" ]] && docker restart "$FULL" >/dev/null + else + ssh -o BatchMode=yes "$M2_SSH" 'FULL=$(docker ps --format "{{.Names}}" | grep -m1 guacamole-full); [ -n "$FULL" ] && docker restart "$FULL" >/dev/null' + fi +fi + +# ---------- summary ---------- +cat <> DONE: $CONTAINER + + desktop container $CONTAINER (IP $DESKTOP_IP on coolify net, host $HOST_ROLE) + volumes ${NAME}-agent-home, ${NAME}-workspace, ${NAME}-m2home + RDP direct ssh -L 3389:$DESKTOP_IP:3389 $(whoami)@$(hostname) → localhost:3389 + RDP user / pass developer / $VNC_PASS +EOF +if ! $SKIP_GUAC; then + cat < Date: Mon, 24 Aug 2026 11:35:16 +0000 Subject: [PATCH 2/4] =?UTF-8?q?propose:=20=C2=A712b=20=E2=80=94=20console-?= =?UTF-8?q?auth=20flow=20+=20Host()=20bug=20fix=20in=20provision.sh?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the Console (ttyd, magic-link flow) subsection to §12b: there is only ONE real console-auth in the fleet (on m2). m2.2 desktops route via m2.2 Traefik → local console-auth socat → m2's console-auth-lan-relay → real console-auth. Documents the two per-desktop sync requirements (Traefik Host + m2's CONSOLE_BASIC_CREDS) and the admin curl to mint magic links. Notes the provision.sh bug (fixed 2026-08-24 on both hosts): the Traefik router Host() was hardcoded to m2o.machinemachine.ai, so /console/ silently 404'd for every m2.2-hosted desktop. provision.sh now reads CONSOLE_PUBLIC_HOST from console-auth/secrets.env. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 5 ++++- PLAYBOOK.md | 34 +++++++++++++++++++++++++++++----- 2 files changed, 33 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c09d32d..65a41de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,9 +2,12 @@ ## [Unreleased] — 2026-08-24 ### Added -- Section 12b: m2o Desktop Provisioning (RDP + cross-host Guacamole) — captures the m2/m2.2 topology, path/DNS gotchas, first-boot `unhealthy` wait, expected WARNs on m2.2, and the socat-relay pattern for wiring m2.2 desktops into m2's Guacamole. +- Section 12b: m2o Desktop Provisioning (RDP + cross-host Guacamole + console/ttyd) — captures the m2/m2.2 topology, path/DNS gotchas, first-boot `unhealthy` wait, expected WARNs on m2.2, the socat-relay pattern for wiring m2.2 desktops into m2's Guacamole, and the console-auth magic-link flow (single real console-auth on m2, m2.2 bridges via socat). - `scripts/launch-m2o-desktop.sh` — idempotent one-command wrapper around `provision.sh` + relay + Guacamole upsert + perm grant. Installed on m2 (`~/m2o/desktop/launch-desktop.sh`) and m2.2 (`~/machinemachine-core/m2o/desktop/launch-desktop.sh`). +### Fixed +- `desktop/provision.sh` (both hosts) — Traefik router `Host()` was hardcoded to `m2o.machinemachine.ai`, breaking `/console/` for every m2.2-hosted desktop (m2o.machinemachine.ai points at m2, not m2.2). Now reads `CONSOLE_PUBLIC_HOST` from `console-auth/secrets.env` (m2.2 has it set to `console.m-2.cc`; m2 defaults to `m2o.machinemachine.ai`). Also patched the existing `console-euroclean.yaml` in place. + ### Authors - Mariusz (operator) — hit the pain, asked for the doc + script - Fable — end-to-end euroclean provisioning + doc/script extraction diff --git a/PLAYBOOK.md b/PLAYBOOK.md index 7736afe..ce84c26 100644 --- a/PLAYBOOK.md +++ b/PLAYBOOK.md @@ -660,14 +660,38 @@ docker inspect -m2o --format '{{.State.Health.Status}}' docker exec -m2o bash -lc 'ps -eo pid,etime,cmd | grep -E "rm -rf|x11vnc|xrdp|guacd" | grep -v grep' ``` -### Expected WARNs on m2.2 (harmless — not bugs) +### Expected WARNs on m2.2 -Because m2.2 doesn't yet run its own Guacamole or console-auth: +Because m2.2 doesn't yet run its own Guacamole and only bridges console-auth via socat: -- `WARN: guacamole-db container not found — create the connection manually` -- `WARN console: console-auth:latest image missing — cred saved to secrets.env` +- `WARN: guacamole-db container not found — create the connection manually` — expected. Wire via `launch-desktop.sh` or the SQL in the next subsection. +- `WARN console: console-auth:latest image missing — cred saved to secrets.env` — expected. m2.2 doesn't host the console-auth image; a socat container named `console-auth` on m2.2's `coolify` network forwards `:8080` to `192.168.31.224:28080` (m2's `console-auth-lan-relay`), so Traefik's `forwardAuth` middleware works transparently. The cred **does still need to reach m2**: append the entry from `~/machinemachine-core/m2o/console-auth/secrets.env` (the `"":"admin:..."` line inside `CONSOLE_BASIC_CREDS`) into m2's `~/m2o/console-auth/secrets.env` and restart `console-auth` on m2. See [Console (ttyd)](#console-ttyd-magic-link-flow) below. -Traefik route for the ttyd web console is written, but `/console/` won't work until console-auth is deployed to m2.2. +### Console (ttyd, magic-link flow) + +The `/console/` browser terminal is served by ttyd inside the desktop container, fronted by Traefik with a `forwardAuth` middleware pointing at `console-auth`. There is only **one real `console-auth`** in the fleet — it runs on m2 (FastAPI, `console-auth:latest`, source at `~/m2o/console-auth/`). Both hosts route to it: + +- **m2 desktops** → m2's Traefik → `console-auth` (same docker network) → ttyd inside the desktop container. Public host: `m2o.machinemachine.ai`. +- **m2.2 desktops** → m2.2's Traefik (public host: `console.m-2.cc`, Cloudflare) → m2.2's socat `console-auth` → m2's `console-auth-lan-relay:28080` → real console-auth on m2 → response back up the chain. ttyd is inside the desktop container on m2.2. + +**Two things must be in sync per desktop** for the browser console to work: + +1. **Traefik router `Host()`** must match the desktop's host (`m2o.machinemachine.ai` on m2, `console.m-2.cc` on m2.2). Old `provision.sh` hardcoded `m2o.machinemachine.ai` for both — **patched 2026-08-24** to read `CONSOLE_PUBLIC_HOST` from `console-auth/secrets.env` (m2.2's secrets sets it to `console.m-2.cc`; m2 leaves the default). +2. **The cred entry** (auto-generated per desktop by provision.sh into the local `secrets.env`) must exist in m2's `CONSOLE_BASIC_CREDS`. On m2 that's automatic (provision.sh redeploys the local console-auth). On m2.2 you must manually copy the `"":"admin:..."` entry into m2's `~/m2o/console-auth/secrets.env` and `docker rm -f console-auth && docker run -d --name console-auth --restart unless-stopped --network coolify --env-file secrets.env console-auth:latest && docker network connect e0o8o8cowkswcwsgs4so48s8 console-auth`. + +**Mint a magic link (admin key, from anywhere on m2):** + +```bash +source ~/m2o/console-auth/secrets.env +CA_IP=$(docker inspect console-auth --format '{{(index .NetworkSettings.Networks "coolify").IPAddress}}') +curl -s -X POST -H "X-Api-Key: $CONSOLE_AUTH_ADMIN_KEY" \ + -H 'Content-Type: application/json' \ + -d '{"desktop":""}' \ + http://$CA_IP:8080/issue +# → returns {"url":"https://m2o.machinemachine.ai/console//?t=..."} +# For m2.2 desktops, swap the host to console.m-2.cc — the token is valid on any host +# (verify uses x-forwarded-host). Link is single-use, 15 min; session cookie is 24 h. +``` ### Wiring an m2.2 desktop into m2's Guacamole From d2b9967baed4c89dec36047824e9d678541d99d1 Mon Sep 17 00:00:00 2001 From: "Mariusz (via Fable)" Date: Mon, 24 Aug 2026 12:20:23 +0000 Subject: [PATCH 3/4] =?UTF-8?q?propose:=20=C2=A712b=20=E2=80=94=20Guacamol?= =?UTF-8?q?e=20"Console"=20button=20+=20CONSOLE=5FHOST=5FOVERRIDES=20+=20f?= =?UTF-8?q?ix=20session-wipe?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Docs the m2o-console-link Guacamole extension (built + deployed this session): tiny JS/CSS extension that adds a Console ↗ button to each row on the Guacamole home page. Click → POST /console-mint/issue-web with the current session token → console-auth validates against the Guacamole REST API and mints a magic link for that desktop. Traefik route /console-mint/* → console-auth was already in place. For the m2.2 desktops the URL must be console.m-2.cc, not the default m2o.machinemachine.ai. Adds CONSOLE_HOST_OVERRIDES env to console-auth (JSON slug → host map) and threads it through _host_for(desktop). launch-desktop.sh now auto-adds the new desktop to this map and redeploys console-auth on m2 when provisioning on m2.2. Also removes the accidental `docker restart guacamole-full` from launch-desktop.sh — Guacamole picks up DB changes on next session, so the restart was gratuitous AND invalidated every active user's cookie (operator saw the connection list briefly "disappear" until re-login). Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 5 ++++ PLAYBOOK.md | 12 ++++++++- scripts/launch-m2o-desktop.sh | 48 +++++++++++++++++++++++++++++------ 3 files changed, 56 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 65a41de..9eaf349 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,11 @@ ### Fixed - `desktop/provision.sh` (both hosts) — Traefik router `Host()` was hardcoded to `m2o.machinemachine.ai`, breaking `/console/` for every m2.2-hosted desktop (m2o.machinemachine.ai points at m2, not m2.2). Now reads `CONSOLE_PUBLIC_HOST` from `console-auth/secrets.env` (m2.2 has it set to `console.m-2.cc`; m2 defaults to `m2o.machinemachine.ai`). Also patched the existing `console-euroclean.yaml` in place. +- `scripts/launch-m2o-desktop.sh` — removed the `docker restart guacamole-full` step from step 5. Guacamole picks up DB changes on next login/session; the restart invalidated every active user's session cookie and made connections briefly appear to "disappear" from the UI. + +### Deployed (side-effects of this session) +- **m2o-console-link Guacamole extension** — the `Console ↗` button on each Guacamole home-screen connection row (source: `~/m2o/guacamole-ext/console-link/` on m2). Extension was already fully written; wasn't installed. Built + deployed via `install-console-link.sh`. +- **`CONSOLE_HOST_OVERRIDES` env in console-auth** — new optional env, JSON `{slug: host}`. `_host_for(desktop)` returns the override or falls back to `PUBLIC_HOST`. Needed so `/issue` and `/issue-web` return `console.m-2.cc` URLs for m2.2 desktops. Patched into `~/m2o/console-auth/app.py` on m2; container rebuilt + redeployed. `launch-desktop.sh` now auto-updates this map when provisioning a new m2.2 desktop. ### Authors - Mariusz (operator) — hit the pain, asked for the doc + script diff --git a/PLAYBOOK.md b/PLAYBOOK.md index ce84c26..faac77b 100644 --- a/PLAYBOOK.md +++ b/PLAYBOOK.md @@ -679,7 +679,17 @@ The `/console/` browser terminal is served by ttyd inside the desktop c 1. **Traefik router `Host()`** must match the desktop's host (`m2o.machinemachine.ai` on m2, `console.m-2.cc` on m2.2). Old `provision.sh` hardcoded `m2o.machinemachine.ai` for both — **patched 2026-08-24** to read `CONSOLE_PUBLIC_HOST` from `console-auth/secrets.env` (m2.2's secrets sets it to `console.m-2.cc`; m2 leaves the default). 2. **The cred entry** (auto-generated per desktop by provision.sh into the local `secrets.env`) must exist in m2's `CONSOLE_BASIC_CREDS`. On m2 that's automatic (provision.sh redeploys the local console-auth). On m2.2 you must manually copy the `"":"admin:..."` entry into m2's `~/m2o/console-auth/secrets.env` and `docker rm -f console-auth && docker run -d --name console-auth --restart unless-stopped --network coolify --env-file secrets.env console-auth:latest && docker network connect e0o8o8cowkswcwsgs4so48s8 console-auth`. -**Mint a magic link (admin key, from anywhere on m2):** +### Guacamole "Console ↗" button (m2o-console-link extension) + +A tiny Guacamole extension (`~/m2o/guacamole-ext/console-link/` on m2) adds a `Console ↗` button next to each connection on the Guacamole home page. Click → POST to `/console-mint/issue-web` with the current Guacamole session token → console-auth validates the caller against the Guacamole REST API, mints a magic link for that desktop, and the browser opens the URL in a new tab. + +- **Traefik route:** `/data/coolify/proxy/dynamic/console-mint.yaml` on m2 exposes `console-auth` at `Host(m2o.machinemachine.ai) && PathPrefix(/console-mint)` with a `stripPrefix` middleware. Same origin as Guacamole, so no CORS. +- **Which rows get a button:** the extension calls `/console-mint/consoles` to get the list of desktop slugs; rows whose name slugifies onto a known desktop show the button. Everything else is hidden. +- **Per-desktop host in the returned URL:** by default `console-auth` returns `https:///console//?t=`. For m2.2-hosted desktops, the URL must instead be on `console.m-2.cc`. This is handled by the `CONSOLE_HOST_OVERRIDES` env in `~/m2o/console-auth/secrets.env` on m2 — a JSON dict `{slug: host}`. `launch-desktop.sh` on m2.2 auto-adds the new desktop to this map and redeploys `console-auth` on m2. +- **Install / update the extension:** on m2, `bash ~/m2o/guacamole-ext/install-console-link.sh`. Rebuilds the JAR, stages it at `/data/coolify/guacamole-home/extensions/` (for the permanent GUACAMOLE_HOME bind mount), copies it into the live container, and reloads the webapp by `touch`ing `web.xml` — no container restart, no session invalidation. +- **Why not a container restart:** the Guacamole entrypoint runs `rm -Rf $HOME/.guacamole` on every container start, which wipes JARs dropped into `~/.guacamole/extensions/`. The bind-mount + `touch web.xml` pattern is the durable install path. + +### Mint a magic link (admin key, from anywhere on m2) ```bash source ~/m2o/console-auth/secrets.env diff --git a/scripts/launch-m2o-desktop.sh b/scripts/launch-m2o-desktop.sh index 09c8aa7..d85a84a 100755 --- a/scripts/launch-m2o-desktop.sh +++ b/scripts/launch-m2o-desktop.sh @@ -201,6 +201,42 @@ else ($CONN_ID,'create-drive-path','true'), ($CONN_ID,'enable-sftp','false'); " + # ---------- 4b. m2.2 host override for console-auth (Guacamole "Console" button) ---------- + if [[ "$HOST_ROLE" == "m2.2" ]]; then + echo ">> [4b/5] recording console host override for '$NAME' → console.m-2.cc" + ssh -o BatchMode=yes "$M2_SSH" bash -s "$NAME" <<'REMOTE' +set -e +NAME="$1" +cd ~/m2o/console-auth +python3 - "$NAME" <<'PY' +import json, pathlib, sys, re +name = sys.argv[1] +p = pathlib.Path("secrets.env") +lines = p.read_text().splitlines() +found = False +for i, l in enumerate(lines): + if l.startswith("CONSOLE_HOST_OVERRIDES="): + raw = l[len("CONSOLE_HOST_OVERRIDES="):].strip() + d = json.loads(raw) if raw else {} + if d.get(name) == "console.m-2.cc": + print("already-set"); sys.exit(0) + d[name] = "console.m-2.cc" + lines[i] = "CONSOLE_HOST_OVERRIDES=" + json.dumps(d, separators=(",", ":")) + found = True + break +if not found: + lines.append('CONSOLE_HOST_OVERRIDES={"' + name + '":"console.m-2.cc"}') +p.write_text("\n".join(lines) + "\n") +print("added") +PY +docker rm -f console-auth >/dev/null 2>&1 || true +docker run -d --name console-auth --restart unless-stopped \ + --network coolify --env-file secrets.env console-auth:latest >/dev/null +docker network connect e0o8o8cowkswcwsgs4so48s8 console-auth 2>/dev/null || true +echo " console-auth redeployed on m2 with '$NAME' host override" +REMOTE + fi + # ---------- 5. perms ---------- echo ">> [5/5] granting perms to: $GRANT_USERS" IFS=',' read -ra USERS <<<"$GRANT_USERS" @@ -218,14 +254,10 @@ else echo " granted: $u_trim" done - # refresh guacamole-full so it picks up the new/updated row immediately - echo " refreshing guacamole-full (restart)" - if [[ "$HOST_ROLE" == "m2" ]]; then - FULL=$(docker ps --format '{{.Names}}' | grep -m1 guacamole-full || true) - [[ -n "$FULL" ]] && docker restart "$FULL" >/dev/null - else - ssh -o BatchMode=yes "$M2_SSH" 'FULL=$(docker ps --format "{{.Names}}" | grep -m1 guacamole-full); [ -n "$FULL" ] && docker restart "$FULL" >/dev/null' - fi + # Guacamole picks up DB changes on next login/session — no restart needed. + # (An earlier version of this script restarted guacamole-full here; that + # invalidated every active user's session and made connections appear to + # "disappear" until they logged back in. Removed 2026-08-24.) fi # ---------- summary ---------- From 11a3366d533ae34d18b2ef2d4fb34466b5bdeee8 Mon Sep 17 00:00:00 2001 From: "Mariusz (via Fable)" Date: Fri, 28 Aug 2026 10:09:36 +0000 Subject: [PATCH 4/4] =?UTF-8?q?propose:=20=C2=A712b=20=E2=80=94=20m2gw=20t?= =?UTF-8?q?enant/agent=20provisioning=20+=20Hermes=20wire?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds `scripts/m2gw-provision-agent.sh` — one idempotent command that creates a tenant, upserts the agent with a configurable route chain, mints a bearer, and (optionally) patches a target desktop's Hermes config.yaml. Runs on m2 or SSH-in from m2.2. Installed on both hosts alongside launch-desktop.sh and in ~/m2-gpt/m2-gpt/scripts/ on m2. Threads a new `--provision-m2gw` flag through launch-desktop.sh so desktop spawn + gateway wire is a single command. New flags --m2gw-primary/--m2gw-fallback/--m2gw-model/--m2gw-principal let the caller pick a different route chain than the default (spark-glm -> deepseek-spark, model=m2gw-spark-glm/glm-5.3-flash). §12b in the playbook gets a "Provisioning the m2-gpt tenant / agent / bearer" subsection with the rationale (per-desktop bearer vs the fleet-wide placeholder baked in by provision.sh), the introspection one-liner for the routes table, and a note that deepseek-v4-3107 isn't deployed today — the fleet standard text route is deepseek-v4-flash-0731. euroclean-m2o brought up end-to-end this session: tenant + agent (chain: spark-glm -> deepseek-spark), per-agent bearer minted, Hermes config on euroclean-m2o updated to use it with model.default=m2gw-spark-glm/glm-5.3-flash. Gateway -> chain -> LLM smoke passed. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 6 + PLAYBOOK.md | 61 ++++++++ scripts/launch-m2o-desktop.sh | 56 ++++++++ scripts/m2gw-provision-agent.sh | 238 ++++++++++++++++++++++++++++++++ 4 files changed, 361 insertions(+) create mode 100755 scripts/m2gw-provision-agent.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 9eaf349..ea76323 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,12 @@ ### Deployed (side-effects of this session) - **m2o-console-link Guacamole extension** — the `Console ↗` button on each Guacamole home-screen connection row (source: `~/m2o/guacamole-ext/console-link/` on m2). Extension was already fully written; wasn't installed. Built + deployed via `install-console-link.sh`. - **`CONSOLE_HOST_OVERRIDES` env in console-auth** — new optional env, JSON `{slug: host}`. `_host_for(desktop)` returns the override or falls back to `PUBLIC_HOST`. Needed so `/issue` and `/issue-web` return `console.m-2.cc` URLs for m2.2 desktops. Patched into `~/m2o/console-auth/app.py` on m2; container rebuilt + redeployed. `launch-desktop.sh` now auto-updates this map when provisioning a new m2.2 desktop. +- **§12b subsection: Provisioning the m2-gpt tenant / agent / bearer** — every desktop's Hermes needs its own bearer (not the fleet-wide placeholder from `~/.m2-gpt-key`) to be a real fleet citizen. Documents the one-command `m2gw-provision-agent.sh` flow and the default route chain (spark-glm → deepseek-spark). +- `scripts/m2gw-provision-agent.sh` — new idempotent one-command wrapper: creates the tenant + agent (with configurable route chain), mints a fresh bearer, and (optionally) patches the target desktop's Hermes `config.yaml` and restarts `hermes-gateway`. Installed on m2 (`~/m2-gpt/m2-gpt/scripts/` **and** `~/m2o/desktop/`) and m2.2 (`~/machinemachine-core/m2o/desktop/`). +- `scripts/launch-m2o-desktop.sh` — new `--provision-m2gw` flag (+ `--m2gw-primary`, `--m2gw-fallback`, `--m2gw-model`, `--m2gw-principal`) makes the desktop spawn + gateway wire a single command. + +### Bring-up done this session +- **euroclean tenant + agent + bearer** on `gpt.machinemachine.ai`, route chain `spark-glm → deepseek-spark`, Hermes on `euroclean-m2o` wired to the per-agent bearer with `model.default: m2gw-spark-glm/glm-5.3-flash`. End-to-end smoke passed (gateway → chain → LLM → response). ### Authors - Mariusz (operator) — hit the pain, asked for the doc + script diff --git a/PLAYBOOK.md b/PLAYBOOK.md index faac77b..13aa2fd 100644 --- a/PLAYBOOK.md +++ b/PLAYBOOK.md @@ -689,6 +689,67 @@ A tiny Guacamole extension (`~/m2o/guacamole-ext/console-link/` on m2) adds a `C - **Install / update the extension:** on m2, `bash ~/m2o/guacamole-ext/install-console-link.sh`. Rebuilds the JAR, stages it at `/data/coolify/guacamole-home/extensions/` (for the permanent GUACAMOLE_HOME bind mount), copies it into the live container, and reloads the webapp by `touch`ing `web.xml` — no container restart, no session invalidation. - **Why not a container restart:** the Guacamole entrypoint runs `rm -Rf $HOME/.guacamole` on every container start, which wipes JARs dropped into `~/.guacamole/extensions/`. The bind-mount + `touch web.xml` pattern is the durable install path. +### Provisioning the m2-gpt tenant / agent / bearer + +Every desktop's Hermes talks to the fleet LLM gateway at `https://gpt.machinemachine.ai/v1`. It authenticates with a **per-agent** bearer (`sk-m2-...`), argon2-hashed at rest — not the fleet-wide `~/.m2-gpt-key` that `provision.sh` bakes in as a placeholder. Without a real per-agent bearer, the desktop is off-fleet: it shares tenancy, budget, and memory with `m2` itself. + +**One command** (installed alongside `launch-desktop.sh` on both hosts, canonical source: `scripts/m2gw-provision-agent.sh` in this repo): + +```bash +# on m2 (or from m2.2 — script SSHes to m2 for gateway ops) +m2gw-provision-agent.sh # sensible defaults +m2gw-provision-agent.sh --primary-route \ + --fallback-route \ + --default-model \ + --principal \ + --hermes-container # patches Hermes config +``` + +Idempotent: existing tenant is reused; existing agent has its `route_bindings` updated; **a fresh bearer is minted every run** (old ones stay `active` — revoke by hand if you want single-key hygiene). If `--hermes-container` matches a container running on this host, the script patches `/home/developer/.hermes/config.yaml` (`model.default` + `model.api_key`) and restarts `hermes-gateway`. + +**Defaults** (chosen 2026-08-28 for the euroclean bring-up): + +| Field | Value | +|-------|-------| +| Primary route (priority 1) | `spark-glm` — self-hosted spark cluster, `glm-5.3-flash` (multimodal, vision-capable) | +| Fallback route (priority 2) | `deepseek-spark` — self-hosted spark, `deepseek-v4-flash-0731` (text) | +| Hermes `model.default` | `m2gw-spark-glm/glm-5.3-flash` | +| Tenant budget | $20/mo | +| Retention | 90 days | + +> Note on model naming: the operator brief asked for `deepseek-v4-3107`. That model id isn't currently deployed on the fleet — the spark cluster serves `deepseek-v4-flash-0731`. Either rename the target model on the upstream, or add a new route entry in `routes` (see `scripts/set-default-route-chain.sh` in the m2-gpt repo for the pattern). + +**All at once via `launch-desktop.sh`:** + +```bash +./launch-desktop.sh --provision-m2gw \ + [--m2gw-primary spark-glm] \ + [--m2gw-fallback deepseek-spark] \ + [--m2gw-model m2gw-spark-glm/glm-5.3-flash] \ + [--m2gw-principal ] +``` + +Runs the same wrapper as step 6 after the desktop is healthy. The bearer is printed once — save it before it scrolls off. + +**Route inventory (read-only glance) — introspect the gateway DB:** + +```bash +# on m2, inside the prod gateway container +PGW=$(docker ps --format '{{.Names}}' | grep '^gateway-akvnse3p7' | head -1) +docker exec $PGW python3 -c " +import os, asyncio, asyncpg +async def m(): + URL=os.environ['M2GW_DATABASE_URL'].replace('postgresql+asyncpg://','postgresql://') + c=await asyncpg.connect(URL) + for r in await c.fetch('select id, provider, base_url, default_params->>' + chr(39) + 'model' + chr(39) + ' as model, usage_scope from routes order by id'): + print(dict(r)) + await c.close() +asyncio.run(m()) +" +``` + +Existing routes at the time of writing: `spark-glm` (glm-5.3-flash, spark cluster), `deepseek-spark` (deepseek-v4-flash-0731, spark), `glm-5.2` / `glm-5.2-zai` / `glm-5.3-zai` (Z.ai proxy), `qwen-fast` / `qwen-local` / `ornith-coding` (local coding models). + ### Mint a magic link (admin key, from anywhere on m2) ```bash diff --git a/scripts/launch-m2o-desktop.sh b/scripts/launch-m2o-desktop.sh index d85a84a..01fc6a8 100755 --- a/scripts/launch-m2o-desktop.sh +++ b/scripts/launch-m2o-desktop.sh @@ -25,6 +25,11 @@ GUAC_DB_ROOT_PASS="guacamole_root_pass" GUAC_DB_NAME="guacamole_db" SKIP_GUAC=false FORCE_RECREATE=false +PROVISION_M2GW=false # opt-in: create tenant/agent/key on gpt.machinemachine.ai +M2GW_PRIMARY_ROUTE="spark-glm" +M2GW_FALLBACK_ROUTE="deepseek-spark" +M2GW_DEFAULT_MODEL="m2gw-spark-glm/glm-5.3-flash" +M2GW_PRINCIPAL="hi@grait.io" usage() { cat < SSH target for m2 (default: m2@192.168.31.224) --skip-guacamole Skip Guacamole wiring (provision + relay only) --force-recreate Rebuild container even if it exists (volumes preserved) + --provision-m2gw Create tenant/agent/bearer on gpt.machinemachine.ai, wire Hermes + --m2gw-primary Route id for priority 1 (default: spark-glm — glm-5.3-flash) + --m2gw-fallback Route id for priority 2 (default: deepseek-spark) + --m2gw-model Hermes model.default (default: m2gw-spark-glm/glm-5.3-flash) + --m2gw-principal Agent principal_ref (default: hi@grait.io) -h, --help This help EOF } @@ -56,6 +66,11 @@ while [[ $# -gt 0 ]]; do --m2-ssh) M2_SSH="$2"; shift 2;; --skip-guacamole) SKIP_GUAC=true; shift;; --force-recreate) FORCE_RECREATE=true; shift;; + --provision-m2gw) PROVISION_M2GW=true; shift;; + --m2gw-primary) M2GW_PRIMARY_ROUTE="$2"; shift 2;; + --m2gw-fallback) M2GW_FALLBACK_ROUTE="$2"; shift 2;; + --m2gw-model) M2GW_DEFAULT_MODEL="$2"; shift 2;; + --m2gw-principal) M2GW_PRINCIPAL="$2"; shift 2;; -h|--help) usage; exit 0;; *) echo "ERROR: unknown option: $1"; usage; exit 1;; esac @@ -260,6 +275,46 @@ REMOTE # "disappear" until they logged back in. Removed 2026-08-24.) fi +# ---------- 6. m2-gpt tenant/agent/bearer + Hermes wire (opt-in) ---------- +M2GW_SUMMARY="" +if $PROVISION_M2GW; then + echo ">> [6/6] m2-gpt: provision tenant+agent+bearer for '$NAME', wire Hermes" + + # Locate the m2gw-provision-agent.sh companion script. + SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + M2GW_SCRIPT="$SCRIPT_DIR/m2gw-provision-agent.sh" + if [[ ! -x "$M2GW_SCRIPT" ]]; then + for cand in "$HOME/m2o/desktop/m2gw-provision-agent.sh" \ + "$HOME/machinemachine-core/m2o/desktop/m2gw-provision-agent.sh"; do + [[ -x "$cand" ]] && { M2GW_SCRIPT="$cand"; break; } + done + fi + if [[ ! -x "$M2GW_SCRIPT" ]]; then + echo " WARN: m2gw-provision-agent.sh not found — install it alongside launch-desktop.sh" + echo " skipping m2-gpt step" + else + if [[ "$HOST_ROLE" == "m2.2" ]]; then + # Runs the gateway ops on m2 via SSH, and wires the Hermes here on m2.2. + # Approach: run the script here (script is host-agnostic and uses --m2-ssh). + "$M2GW_SCRIPT" "$NAME" \ + --primary-route "$M2GW_PRIMARY_ROUTE" \ + --fallback-route "$M2GW_FALLBACK_ROUTE" \ + --default-model "$M2GW_DEFAULT_MODEL" \ + --principal "$M2GW_PRINCIPAL" \ + --hermes-container "$CONTAINER" \ + --m2-ssh "$M2_SSH" + else + "$M2GW_SCRIPT" "$NAME" \ + --primary-route "$M2GW_PRIMARY_ROUTE" \ + --fallback-route "$M2GW_FALLBACK_ROUTE" \ + --default-model "$M2GW_DEFAULT_MODEL" \ + --principal "$M2GW_PRINCIPAL" \ + --hermes-container "$CONTAINER" + fi + M2GW_SUMMARY=" m2-gpt tenant/agent $NAME chain=$M2GW_PRIMARY_ROUTE → $M2GW_FALLBACK_ROUTE model=$M2GW_DEFAULT_MODEL" + fi +fi + # ---------- summary ---------- cat < [OPTIONS] +# +# Options: +# --primary-route Route for priority 1 (default: spark-glm — glm-5.3-flash) +# --fallback-route Route for priority 2 (default: deepseek-spark — deepseek text) +# --extra-fallback Additional route at priority 3 (default: none) +# --default-model Hermes model.default (default: m2gw-spark-glm/glm-5.3-flash) +# --principal Principal ref for the agent (default: hi@grait.io) +# --budget-usd Tenant monthly budget in USD (default: 20) +# --retention-days Tenant memory retention (default: 90) +# --hermes-container Docker container name of the desktop to wire (default: -m2o if it exists on THIS host) +# --skip-hermes Do not update any container's Hermes config +# --m2-ssh SSH target for m2 (only used when NOT on m2) (default: m2@192.168.31.224) +# -h, --help +# +# Prints the minted bearer once. Copy it before scrolling. + +set -euo pipefail + +# -------- defaults -------- +PRIMARY_ROUTE="spark-glm" +FALLBACK_ROUTE="deepseek-spark" +EXTRA_FALLBACK="" +DEFAULT_MODEL="m2gw-spark-glm/glm-5.3-flash" +PRINCIPAL="hi@grait.io" +BUDGET_USD="20" +RETENTION_DAYS="90" +HERMES_CONTAINER="" +SKIP_HERMES=false +M2_SSH="m2@192.168.31.224" + +# -------- parse -------- +[[ $# -lt 1 ]] && { grep -E '^# ' "$0" | sed 's/^# \{0,1\}//'; exit 1; } +case "$1" in -h|--help) grep -E '^# ' "$0" | sed 's/^# \{0,1\}//'; exit 0;; esac +SLUG="$1"; shift +[[ "$SLUG" =~ ^[a-z][a-z0-9-]*$ ]] || { echo "ERROR: slug must be lowercase alnum/hyphen"; exit 1; } + +while [[ $# -gt 0 ]]; do + case "$1" in + --primary-route) PRIMARY_ROUTE="$2"; shift 2;; + --fallback-route) FALLBACK_ROUTE="$2"; shift 2;; + --extra-fallback) EXTRA_FALLBACK="$2"; shift 2;; + --default-model) DEFAULT_MODEL="$2"; shift 2;; + --principal) PRINCIPAL="$2"; shift 2;; + --budget-usd) BUDGET_USD="$2"; shift 2;; + --retention-days) RETENTION_DAYS="$2"; shift 2;; + --hermes-container) HERMES_CONTAINER="$2"; shift 2;; + --skip-hermes) SKIP_HERMES=true; shift;; + --m2-ssh) M2_SSH="$2"; shift 2;; + -h|--help) grep -E '^# ' "$0" | sed 's/^# \{0,1\}//'; exit 0;; + *) echo "ERROR: unknown option: $1"; exit 1;; + esac +done + +# Detect if we're already on m2. +ON_M2=false +case "$(hostname)" in m2|m2-main|m2main) ON_M2=true;; esac + +# -------- helper: run a python snippet inside the prod gateway container -------- +run_in_gateway() { + local script="$1" + local remote_cmd=' +PGW=$(docker ps --format "{{.Names}}" | grep "^gateway-akvnse3p7" | head -1) +[ -n "$PGW" ] || { echo "ERROR: prod gateway container not found"; exit 1; } +python3 - < "$tmpname" + local PGW + PGW=$(docker ps --format '{{.Names}}' | grep '^gateway-akvnse3p7' | head -1) + [[ -n "$PGW" ]] || { echo "ERROR: prod gateway container not found"; rm -f "$tmpname"; exit 1; } + docker cp "$tmpname" "$PGW:$tmpname" >/dev/null + docker exec "$PGW" python3 "$tmpname" + docker exec "$PGW" rm -f "$tmpname" || true + rm -f "$tmpname" + else + ssh -o BatchMode=yes "$M2_SSH" \ + "cat > $tmpname && PGW=\$(docker ps --format '{{.Names}}' | grep '^gateway-akvnse3p7' | head -1) && \ + docker cp $tmpname \$PGW:$tmpname >/dev/null && docker exec \$PGW python3 $tmpname && \ + docker exec \$PGW rm -f $tmpname && rm -f $tmpname" <<<"$py" + fi +} + +# ---------- 1. tenant (idempotent) ---------- +echo ">> [1/4] ensure tenant '$SLUG'" +TENANT_PY="import os, asyncio, asyncpg +SLUG, ROUTE, BUDGET, RETENTION, PRINCIPAL = '$SLUG', '$PRIMARY_ROUTE', $BUDGET_USD, $RETENTION_DAYS, '$PRINCIPAL' +async def main(): + URL = os.environ['M2GW_DATABASE_URL'].replace('postgresql+asyncpg://', 'postgresql://') + c = await asyncpg.connect(URL) + exists = await c.fetchval('select 1 from tenants where id=\$1', SLUG) + if exists: + print(f' tenant {SLUG} already exists') + else: + await c.execute( + \"\"\"insert into tenants (id, display_name, default_retention_days, + default_budget_monthly_usd, default_route_id, status, created_by, + rate_limit_per_minute, compaction_policy) + values (\$1, \$1, \$2, \$3, \$4, 'active', \$5, 60, '{}')\"\"\", + SLUG, RETENTION, BUDGET, ROUTE, PRINCIPAL, + ) + print(f' created tenant {SLUG} (default_route={ROUTE}, budget=\${BUDGET}/mo, retention={RETENTION}d)') + await c.close() +asyncio.run(main()) +" +run_in_gateway_file "$TENANT_PY" + +# ---------- 2. agent (idempotent — update route bindings if it exists) ---------- +echo ">> [2/4] ensure agent '$SLUG' with route chain [${PRIMARY_ROUTE}${FALLBACK_ROUTE:+, ${FALLBACK_ROUTE}}${EXTRA_FALLBACK:+, ${EXTRA_FALLBACK}}]" +CHAIN_JSON="[{\"route_id\":\"$PRIMARY_ROUTE\",\"priority\":1}" +[[ -n "$FALLBACK_ROUTE" ]] && CHAIN_JSON="$CHAIN_JSON,{\"route_id\":\"$FALLBACK_ROUTE\",\"priority\":2}" +[[ -n "$EXTRA_FALLBACK" ]] && CHAIN_JSON="$CHAIN_JSON,{\"route_id\":\"$EXTRA_FALLBACK\",\"priority\":3}" +CHAIN_JSON="$CHAIN_JSON]" + +AGENT_PY="import os, asyncio, asyncpg, json +SLUG, PRINCIPAL, CHAIN = '$SLUG', '$PRINCIPAL', json.loads('''$CHAIN_JSON''') +TOOLS = ['m2.memory.query', 'm2.memory.write'] +async def main(): + URL = os.environ['M2GW_DATABASE_URL'].replace('postgresql+asyncpg://', 'postgresql://') + c = await asyncpg.connect(URL) + exists = await c.fetchval('select 1 from agents where id=\$1', SLUG) + if exists: + await c.execute('update agents set route_bindings=\$1::jsonb, principal_ref=\$2 where id=\$3', + json.dumps(CHAIN), PRINCIPAL, SLUG) + print(f' updated agent {SLUG} chain={[b[\"route_id\"] for b in CHAIN]}') + else: + await c.execute( + \"\"\"insert into agents (id, tenant_id, principal_ref, subconscious_tools, + route_bindings, config_version, status, subconscious_config) + values (\$1, \$1, \$2, \$3::jsonb, \$4::jsonb, 'v1', 'active', '{}'::jsonb)\"\"\", + SLUG, PRINCIPAL, json.dumps(TOOLS), json.dumps(CHAIN), + ) + print(f' created agent {SLUG} chain={[b[\"route_id\"] for b in CHAIN]}') + await c.close() +asyncio.run(main()) +" +run_in_gateway_file "$AGENT_PY" + +# ---------- 3. bearer (always mint fresh) ---------- +echo ">> [3/4] mint bearer for '$SLUG'" +MINT_PY="import os, asyncio, asyncpg, uuid, sys +from gateway.admin_api.keys import _generate_bearer_key, _hash_bearer, _compute_lookup_idx +SLUG = '$SLUG' +async def main(): + URL = os.environ['M2GW_DATABASE_URL'].replace('postgresql+asyncpg://', 'postgresql://') + c = await asyncpg.connect(URL) + row = await c.fetchrow('select id, tenant_id from agents where id=\$1', SLUG) + if not row: + print(f'ERROR: agent \\\"{SLUG}\\\" not found', file=sys.stderr); await c.close(); sys.exit(1) + raw = _generate_bearer_key() + kid = str(uuid.uuid4()) + await c.execute( + 'insert into api_keys (id, tenant_id, agent_id, bearer_hash, bearer_lookup_idx, status) ' + \"values (\$1, \$2, \$3, \$4, \$5, 'active')\", + kid, row['tenant_id'], SLUG, _hash_bearer(raw), _compute_lookup_idx(raw), + ) + print(f'BEARER={raw}') + print(f'KEY_ID={kid}') + await c.close() +asyncio.run(main()) +" +MINT_OUT="$(run_in_gateway_file "$MINT_PY")" +echo "$MINT_OUT" +BEARER="$(printf '%s\n' "$MINT_OUT" | awk -F= '/^BEARER=/{print $2; exit}')" +KEY_ID="$(printf '%s\n' "$MINT_OUT" | awk -F= '/^KEY_ID=/{print $2; exit}')" +[[ -n "$BEARER" ]] || { echo "ERROR: mint failed"; exit 1; } + +# ---------- 4. Hermes wire (optional) ---------- +if $SKIP_HERMES; then + echo ">> [4/4] --skip-hermes — done" +else + [[ -z "$HERMES_CONTAINER" ]] && HERMES_CONTAINER="${SLUG}-m2o" + if ! docker inspect "$HERMES_CONTAINER" >/dev/null 2>&1; then + echo ">> [4/4] container '$HERMES_CONTAINER' not on this host — copy the bearer manually" + echo " (or re-run with --hermes-container on the host where the desktop lives)" + else + echo ">> [4/4] wire Hermes in container '$HERMES_CONTAINER'" + docker exec "$HERMES_CONTAINER" bash -c " +set -e +cd /home/developer/.hermes +cp config.yaml config.yaml.bak.\$(date +%s) +python3 - </dev/null +sleep 2 +supervisorctl status hermes-gateway | tr -s ' ' +" + fi +fi + +# ---------- summary ---------- +cat <> DONE — tenant + agent + bearer for '$SLUG' on gpt.machinemachine.ai + + tenant $SLUG + agent $SLUG (principal=$PRINCIPAL) + route chain ${PRIMARY_ROUTE}${FALLBACK_ROUTE:+ → ${FALLBACK_ROUTE}}${EXTRA_FALLBACK:+ → ${EXTRA_FALLBACK}} + default model $DEFAULT_MODEL + key_id $KEY_ID + bearer $BEARER + ^^ save this now — the raw key is NOT stored (argon2-hashed at rest) +EOF +if ! $SKIP_HERMES && docker inspect "${HERMES_CONTAINER:-${SLUG}-m2o}" >/dev/null 2>&1; then + echo " Hermes config.yaml on ${HERMES_CONTAINER:-${SLUG}-m2o} updated, hermes-gateway restarted" +fi