diff --git a/docs/terminal-streaming.md b/docs/terminal-streaming.md new file mode 100644 index 00000000..e38cb4ba --- /dev/null +++ b/docs/terminal-streaming.md @@ -0,0 +1,117 @@ +# Terminal Snapshot Streaming Contracts + +Codeman exposes three coordinated sources for restoring a terminal: + +1. a bounded rendered tmux history page; +2. a current-pane HTTP snapshot; and +3. live `session:terminal` events arriving over SSE, followed by WebSocket output. + +HTTP content encoding remains lossless: Fastify negotiates Brotli or gzip, and the browser +decompresses each response incrementally. The contracts let a client decouple history +download from terminal parsing and avoid requesting the full retained conversation. + +## Cursor Contract + +Every Session owns a terminal stream id and a lexicographically monotonic cursor: + +```ts +interface TerminalCursor { + stream: string; + generation: number; + start: number; + end: number; +} +``` + +- `stream` changes when the in-memory Session is recreated. +- `generation` increases when the retained terminal buffer is replaced or cleared. +- `start` and `end` are UTF-16 string offsets within that generation. +- A live event covers `[start, end)`. +- A snapshot cursor identifies the live-output boundary represented by the snapshot. + +A client can queue live events while a snapshot is loading. Once the snapshot finishes: + +- events ending at or before the snapshot boundary are discarded; +- events starting at or after the boundary are replayed; +- a batch crossing the boundary replays only its uncovered suffix; +- events from older generations or a replaced stream are discarded; +- events from a newer generation are replayed in full. + +This replaces the previous empty-buffer heuristic, which could either duplicate an Ink +redraw or discard a prompt emitted during the fetch. + +## History Page Contract + +`GET /api/sessions/:id/terminal?historyPage=1&lines=400&format=stream` returns +one bounded physical-row slice of retained tmux history. It never includes the visible pane. + +Page headers: + +- `X-Codeman-History-Start` / `End`: half-open absolute row range from the + oldest retained tmux history row. +- `X-Codeman-History-Total`: retained history rows at capture time. +- `X-Codeman-History-More-Before` / `More-After`: navigable directions. +- `X-Codeman-History-Origin`: opaque pane/history-origin fingerprint. + +Omitting `before`/`after` returns the newest page. `before=` walks toward +older output; `after=` walks toward newer output. The browser requests 400 +rows per page; the server accepts and clamps explicit sizes from 100–2,000 +physical rows. Page capture deliberately does not use tmux `-J`, so each page +coordinate remains one physical tmux row. + +The tmux commands run asynchronously, so page capture does not block terminal +input, SSE flushes, or other sessions while the subprocess returns. + +If tmux page capture fails, the endpoint falls back to the newest 1 MB of the +PTY buffer without page metadata. The client then disables row paging for that +load because byte offsets cannot safely substitute for tmux row coordinates. + +`X-Codeman-History-Origin` changes when the pane or oldest retained rows change. +The client rejects a page with a different origin instead of stitching rows +across tmux eviction or pane replacement. + +## HTTP Snapshot Contract + +`GET /api/sessions/:id/terminal?format=stream` returns raw terminal text rather than the +JSON API envelope. Snapshot metadata is carried in `X-Codeman-Terminal-*` headers. The +existing JSON response remains available for old clients and as a frontend fallback. + +`latest=1` projects the same endpoint down to the current rendered tmux pane. It can +provide a bounded first paint while the newest history page loads independently. The +latest response supplies the authoritative PTY cursor for live-event reconciliation. + +`full=1` remains available for legacy callers that explicitly require one complete tmux +capture, but the browser no longer uses it during startup or session switching. + +The stream response is `no-store`. It may still be reconstructed from tmux history or a +visible pane capture, so cursor offsets order live events; they are not byte offsets into +the normalized response body. + +## Live Transport Handoff + +SSE and WebSocket can use the same per-tab transport id (`clientId:tabNonce`). When a +WebSocket opens with that id, the server flushes that session's pending SSE batch before +suppressing SSE terminal events for the tab. The WebSocket then becomes the only live +terminal transport for that session. + +Active WebSocket output uses a viewport-aware, lossless micro-batch. Desktop and +tablet connections retain the 8 ms / 8 KB low-latency path. A connection that +announces a mobile viewport groups raw PTY output for at most 50 ms and emits +approximately 16 KB frames. The wider phone window combines adjacent decorative +redraws before adding one DEC-2026 synchronized-update pair, reducing full xterm +paints without discarding terminal data. + +Bulk data is split on UTF-8- and ANSI-safe boundaries before each payload is +wrapped. A single control sequence, OSC/DCS string, or application-owned +synchronized update remains indivisible even when it exceeds the viewport target; +terminal correctness takes priority over a hard packet limit. If a PTY event ends +inside a control sequence, the connection emits its safe prefix and carries the +incomplete suffix into the next event rather than inserting a synchronization +marker inside the command. + +For an intentional disconnect, a client may send a handoff frame and wait briefly for +an acknowledgement. The server flushes pending WebSocket output, detaches its Session +listeners, discards any overlapping SSE batch while the tab is still suppressed, then +resumes SSE and acknowledges the handoff. An unexpected close resumes SSE with a +targeted `session:needsRefresh` event so the client can reload an authoritative snapshot +rather than risk a missing transport interval. diff --git a/src/mux-interface.ts b/src/mux-interface.ts index d6e58d4e..1927dce6 100644 --- a/src/mux-interface.ts +++ b/src/mux-interface.ts @@ -132,6 +132,31 @@ export interface PaneCaptureOptions { maxCaptureBytes?: number; } +/** A stable physical-row slice of a tmux pane's retained history. */ +export interface PaneHistoryPage { + /** ANSI-styled physical rows separated by CRLF, excluding the visible pane. */ + buffer: string; + /** Inclusive absolute row offset from the oldest retained history row. */ + start: number; + /** Exclusive absolute row offset from the oldest retained history row. */ + end: number; + /** Retained history rows at capture time, excluding the visible pane. */ + total: number; + hasMoreBefore: boolean; + hasMoreAfter: boolean; + /** Changes when the pane or oldest retained history changes. */ + origin: string; +} + +export interface PaneHistoryPageOptions { + /** Maximum physical tmux rows to return. */ + limit: number; + /** Fetch the page ending immediately before this absolute history row. */ + before?: number; + /** Fetch the page beginning at this absolute history row. */ + after?: number; +} + /** * Terminal multiplexer interface. * @@ -271,4 +296,7 @@ export interface TerminalMultiplexer extends EventEmitter { * Pass `{ fullHistory: true }` to capture the entire scrollback (COD-47). */ captureActivePaneBuffer?(muxName: string, opts?: PaneCaptureOptions): string | null; + + /** Capture one bounded physical-row page from the active pane's retained history. */ + captureActivePaneHistoryPage?(muxName: string, opts: PaneHistoryPageOptions): Promise; } diff --git a/src/session.ts b/src/session.ts index 98e05940..c60d7f0b 100644 --- a/src/session.ts +++ b/src/session.ts @@ -99,6 +99,19 @@ export type { RalphTrackerState, RalphTodoItem, ActiveBashTool } from './types.j export type ResizeViewportType = 'mobile' | 'tablet' | 'desktop'; +/** + * Orders terminal output within one in-memory Session. + * + * Offsets use JavaScript string (UTF-16) units so the browser can slice an + * overlapping SSE batch with the same values without a byte conversion. + */ +export interface TerminalCursor { + stream: string; + generation: number; + start: number; + end: number; +} + /** Line buffer flush interval (100ms) - forces processing of partial lines */ const LINE_BUFFER_FLUSH_INTERVAL = 100; @@ -310,6 +323,9 @@ export class Session extends EventEmitter { private _respawnBlocked = false; // Use BufferAccumulator for hot-path buffers to reduce GC pressure private _terminalBuffer = new BufferAccumulator(MAX_TERMINAL_BUFFER_SIZE, TERMINAL_BUFFER_TRIM_SIZE); + private readonly _terminalStreamId = uuidv4(); + private _terminalGeneration = 0; + private _terminalCursorEnd = 0; private _textOutput = new BufferAccumulator(MAX_TEXT_OUTPUT_SIZE, TEXT_OUTPUT_TRIM_SIZE); private _errorBuffer: string = ''; private _lastActivityAt: number; @@ -658,6 +674,15 @@ export class Session extends EventEmitter { return this._terminalBuffer.length; } + get terminalCursor(): TerminalCursor { + return { + stream: this._terminalStreamId, + generation: this._terminalGeneration, + start: Math.max(0, this._terminalCursorEnd - this._terminalBuffer.length), + end: this._terminalCursorEnd, + }; + } + get textOutput(): string { return this._textOutput.value; } @@ -1408,10 +1433,10 @@ export class Session extends EventEmitter { }); } - // BufferAccumulator handles auto-trimming when max size exceeded - this._terminalBuffer.append(data); + // BufferAccumulator handles auto-trimming when max size exceeded. + const cursor = this._appendTerminalBuffer(data); this._lastActivityAt = Date.now(); - this.emit('terminal', data); + this.emit('terminal', data, cursor); this.emit('output', data); } @@ -1546,7 +1571,7 @@ export class Session extends EventEmitter { // Clean the buffer - remove mux init junk before actual content // Strip: cursor movement (\x1b[nA/B/C/D), positioning (\x1b[n;nH), // clear screen (\x1b[2J), scroll region (\x1b[n;nr), and whitespace - this._terminalBuffer.set(bufferValue.replace(LEADING_ANSI_WHITESPACE_PATTERN, '')); + this._replaceTerminalBuffer(bufferValue.replace(LEADING_ANSI_WHITESPACE_PATTERN, '')); // Signal client to refresh this.emit('clearTerminal'); } @@ -1901,7 +1926,7 @@ export class Session extends EventEmitter { if (!isRestored) { setTimeout(() => { if (this.ptyProcess) { - this._terminalBuffer.clear(); + this._clearTerminalBuffer(); this.ptyProcess.write('clear\n'); } }, 100); @@ -2115,7 +2140,7 @@ export class Session extends EventEmitter { private _resetBuffers(): void { this._status = 'busy'; - this._terminalBuffer.clear(); + this._clearTerminalBuffer(); this._textOutput.clear(); this._errorBuffer = ''; this._messages = []; @@ -2863,7 +2888,7 @@ export class Session extends EventEmitter { assignTask(taskId: string): void { this._currentTaskId = taskId; this._status = 'busy'; - this._terminalBuffer.clear(); + this._clearTerminalBuffer(); this._textOutput.clear(); this._errorBuffer = ''; this._messages = []; @@ -2889,7 +2914,7 @@ export class Session extends EventEmitter { } clearBuffers(): void { - this._terminalBuffer.clear(); + this._clearTerminalBuffer(); this._textOutput.clear(); this._errorBuffer = ''; this._messages = []; @@ -2897,4 +2922,28 @@ export class Session extends EventEmitter { this._ralphTracker.clear(); this._taskCache.clear(); } + + private _appendTerminalBuffer(data: string): TerminalCursor { + const start = this._terminalCursorEnd; + this._terminalBuffer.append(data); + this._terminalCursorEnd += data.length; + return { + stream: this._terminalStreamId, + generation: this._terminalGeneration, + start, + end: this._terminalCursorEnd, + }; + } + + private _replaceTerminalBuffer(data: string): void { + this._terminalBuffer.set(data); + this._terminalGeneration++; + this._terminalCursorEnd = data.length; + } + + private _clearTerminalBuffer(): void { + this._terminalBuffer.clear(); + this._terminalGeneration++; + this._terminalCursorEnd = 0; + } } diff --git a/src/tmux-manager.ts b/src/tmux-manager.ts index ca29478e..997550be 100644 --- a/src/tmux-manager.ts +++ b/src/tmux-manager.ts @@ -22,10 +22,12 @@ */ import { EventEmitter } from 'node:events'; -import { execSync, exec } from 'node:child_process'; +import { execSync, exec, execFile } from 'node:child_process'; +import { createHash } from 'node:crypto'; import { promisify } from 'node:util'; const execAsync = promisify(exec); +const execFileAsync = promisify(execFile); import { existsSync, readFileSync, mkdirSync } from 'node:fs'; import { writeFile, rename } from 'node:fs/promises'; import { dirname } from 'node:path'; @@ -77,6 +79,8 @@ import type { CreateSessionOptions, RespawnPaneOptions, PaneCaptureOptions, + PaneHistoryPage, + PaneHistoryPageOptions, } from './mux-interface.js'; import { decideReconnect, @@ -100,6 +104,9 @@ import { DEFAULT_TMUX_HISTORY_LIMIT, DEFAULT_TERMINAL_BUFFER_MAX_BYTES } from '. * capture must be allowed to exceed the final payload size. */ const FULL_HISTORY_CAPTURE_SLACK_BYTES = 8 * 1024 * 1024; +const HISTORY_PAGE_MAX_BUFFER_BYTES = 4 * 1024 * 1024; +const HISTORY_ORIGIN_SAMPLE_ROWS = 3; +const HISTORY_COMMAND_MAX_BUFFER_BYTES = 64 * 1024; /** Delay after tmux session creation — enough for detached tmux to be queryable */ const TMUX_CREATION_WAIT_MS = 100; @@ -501,6 +508,78 @@ export function normalizeScrollbackEol(buffer: string): string { return buffer.replace(/\r?\n/g, '\r\n'); } +export function resolveHistoryPageRange( + historySize: number, + options: PaneHistoryPageOptions +): Omit { + const total = Math.max(0, Math.trunc(Number.isFinite(historySize) ? historySize : 0)); + const limit = Math.max(1, Math.trunc(Number.isFinite(options.limit) ? options.limit : 1)); + + let start: number; + let end: number; + if (Number.isFinite(options.after)) { + start = Math.max(0, Math.min(total, Math.trunc(options.after as number))); + end = Math.min(total, start + limit); + } else { + end = Number.isFinite(options.before) ? Math.max(0, Math.min(total, Math.trunc(options.before as number))) : total; + start = Math.max(0, end - limit); + } + + return { + start, + end, + total, + hasMoreBefore: start > 0, + hasMoreAfter: end < total, + }; +} + +export type TmuxHistoryCommandRunner = (args: string[], maxBuffer: number) => Promise; + +/** + * Capture one stable history page through an injected tmux command runner. + * Keeping the command sequence here makes the paging algorithm independently + * testable without allowing tests to reach a real tmux server. + */ +export async function captureHistoryPageWithRunner( + target: string, + options: PaneHistoryPageOptions, + run: TmuxHistoryCommandRunner +): Promise { + const historyText = ( + await run(['display-message', '-p', '-t', target, '#{history_size}'], HISTORY_COMMAND_MAX_BUFFER_BYTES) + ).trim(); + const historySize = parseInt(historyText, 10); + if (!Number.isSafeInteger(historySize) || historySize < 0) return null; + + const range = resolveHistoryPageRange(historySize, options); + let buffer = ''; + if (range.end > range.start) { + const startLine = range.start - historySize; + const endLine = range.end - historySize - 1; + const captured = await run( + ['capture-pane', '-p', '-e', '-S', String(startLine), '-E', String(endLine), '-t', target], + HISTORY_PAGE_MAX_BUFFER_BYTES + ); + // tmux terminates stdout with one newline in addition to the captured row + // separators. Remove exactly that terminator so trailing blank rows still + // count toward the page's physical-row coordinates. + buffer = normalizeScrollbackEol(captured.endsWith('\n') ? captured.slice(0, -1) : captured); + } + + let originSample = ''; + if (historySize > 0) { + const sampleEndLine = -Math.max(1, historySize - HISTORY_ORIGIN_SAMPLE_ROWS + 1); + originSample = await run( + ['capture-pane', '-p', '-e', '-S', String(-historySize), '-E', String(sampleEndLine), '-t', target], + HISTORY_COMMAND_MAX_BUFFER_BYTES + ); + } + const origin = createHash('sha256').update(target).update('\0').update(originSample).digest('hex').slice(0, 24); + + return { buffer, origin, ...range }; +} + export function formatPaneSnapshot( lines: string[], geometry: { cols: number; rows: number; cursorX: number; cursorY: number } @@ -3079,6 +3158,74 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer { } } + /** + * Capture a bounded physical-row slice of tmux history. + * + * Page coordinates are absolute offsets from the oldest retained history + * row. Unlike full-history restoration this deliberately omits `-J`: one + * captured tmux row must remain one page coordinate so adjacent pages can be + * composed without byte or soft-wrap ambiguity. + */ + private async runTmuxHistoryCommand(args: string[], maxBuffer: number): Promise { + const { stdout } = await execFileAsync('tmux', ['-L', this.tmuxSocket, ...args], { + encoding: 'utf8', + timeout: EXEC_TIMEOUT_MS, + maxBuffer, + }); + return stdout; + } + + async capturePaneHistoryPage( + muxName: string, + paneTarget: string | undefined, + opts: PaneHistoryPageOptions + ): Promise { + if (IS_TEST_MODE) return null; + const target = resolveTmuxPaneTarget(muxName, paneTarget); + if (!target) { + console.error('[TmuxManager] Invalid pane target in capturePaneHistoryPage:', { + muxName, + paneTarget, + }); + return null; + } + + try { + return await captureHistoryPageWithRunner(target, opts, (args, maxBuffer) => + this.runTmuxHistoryCommand(args, maxBuffer) + ); + } catch (err) { + if ((err as NodeJS.ErrnoException)?.code === 'ENOBUFS') { + console.error('[TmuxManager] History page exceeded its bounded capture buffer'); + } else { + console.error('[TmuxManager] Failed to capture pane history page:', err); + } + return null; + } + } + + async captureActivePaneHistoryPage(muxName: string, opts: PaneHistoryPageOptions): Promise { + if (IS_TEST_MODE) return null; + if (!isValidMuxName(muxName)) { + console.error('[TmuxManager] Invalid session name in captureActivePaneHistoryPage:', muxName); + return null; + } + + try { + const output = ( + await this.runTmuxHistoryCommand( + ['list-panes', '-t', muxName, '-F', '#{pane_id}:#{pane_active}'], + HISTORY_COMMAND_MAX_BUFFER_BYTES + ) + ).trim(); + const target = resolveActivePaneTarget(output); + return target ? await this.capturePaneHistoryPage(muxName, target, opts) : null; + } catch (err) { + console.error('[TmuxManager] Failed to resolve active pane for history capture:', err); + return null; + } + } + /** * Start piping pane output to a file using tmux pipe-pane. * Only pipes output direction (-O) to avoid echoing input. diff --git a/src/web/ports/event-port.ts b/src/web/ports/event-port.ts index 1298dfb7..a3d937cb 100644 --- a/src/web/ports/event-port.ts +++ b/src/web/ports/event-port.ts @@ -3,12 +3,12 @@ * Route modules that need to notify the frontend depend on this port. */ -import type { BackgroundTask } from '../../session.js'; +import type { BackgroundTask, TerminalCursor } from '../../session.js'; export interface EventPort { broadcast(event: string, data: unknown): void; sendPushNotifications(event: string, data: Record): void; - batchTerminalData(sessionId: string, data: string): void; + batchTerminalData(sessionId: string, data: string, cursor?: TerminalCursor): void; broadcastSessionStateDebounced(sessionId: string): void; batchTaskUpdate(sessionId: string, task: BackgroundTask): void; getSseClientCount(): number; diff --git a/src/web/routes/session-routes.ts b/src/web/routes/session-routes.ts index 20636f8a..6fb6f791 100644 --- a/src/web/routes/session-routes.ts +++ b/src/web/routes/session-routes.ts @@ -11,6 +11,7 @@ import { existsSync, statSync, mkdirSync, writeFileSync } from 'node:fs'; import { execFile } from 'node:child_process'; import fs from 'node:fs/promises'; import { randomBytes } from 'node:crypto'; +import { Readable } from 'node:stream'; import { ApiErrorCode, createErrorResponse, @@ -142,6 +143,23 @@ const ERASE_SCROLLBACK_PATTERN = /\x1b\[3J/g; // strip existed can still carry them; strip on replay for parity. // eslint-disable-next-line no-control-regex const MOUSE_TRACKING_PATTERN = /\x1b\[\?(?:1000|1001|1002|1003|1005|1006|1007)[hl]/g; +// Historical synchronized-output markers can span network chunks. Strip them +// before streaming so incremental replay cannot inherit a half-open sync block. +// eslint-disable-next-line no-control-regex +const DEC_SYNC_OUTPUT_PATTERN = /\x1b\[\?2026[hl]/g; +const TERMINAL_STREAM_CHUNK_BYTES = 32 * 1024; +const TERMINAL_HISTORY_PAGE_FALLBACK_BYTES = 1024 * 1024; + +function createTerminalSnapshotStream(data: string): Readable { + const bytes = Buffer.from(data, 'utf8'); + return Readable.from( + (function* () { + for (let offset = 0; offset < bytes.length; offset += TERMINAL_STREAM_CHUNK_BYTES) { + yield bytes.subarray(offset, offset + TERMINAL_STREAM_CHUNK_BYTES); + } + })() + ); +} /** * Strip redundant Ink spinner/status-bar redraw frames from the terminal buffer. @@ -1568,23 +1586,100 @@ export function registerSessionRoutes( // ========== Get Terminal Buffer ========== // Query params: - // tail= - Only return last N bytes (faster initial load) - // full=1 - Full page reload: replay the entire tmux scrollback (COD-47) - app.get('/api/sessions/:id/terminal', async (req) => { + // tail= - Only return last N bytes (faster initial load) + // full=1 - Legacy full reload: replay the entire tmux scrollback (COD-47) + // latest=1 - Current rendered pane only, for a stable first frame + // historyPage=1 - One bounded physical-row page from retained tmux history + // before= - Page backward from this absolute retained-history row + // after= - Page forward from this absolute retained-history row + // lines= - Requested history-page size (server-clamped) + // format=stream - Raw streamed terminal body with metadata in response headers + app.get('/api/sessions/:id/terminal', async (req, reply) => { const { id } = req.params as { id: string }; - const query = req.query as { tail?: string; full?: string }; + const query = req.query as { + tail?: string; + full?: string; + latest?: string; + historyPage?: string; + before?: string; + after?: string; + lines?: string; + format?: string; + }; const session = findSessionOrFail(ctx, id, req); - // `full=1` is the EXPLICIT full-reload signal (COD-47): the browser reloaded - // the page and wants the whole scroll history back, so we capture the ENTIRE - // tmux scrollback and the user gets back history that scrolled off Codeman's - // byte buffer. Requests WITHOUT it — tab switches (`tail=`) and the legacy - // no-param callers (response-viewer fallback, clearTerminal refresh) — keep - // the fast visible-frame capture. - const tailBytes = query.tail ? parseInt(query.tail, 10) : 0; + // `full=1` remains an explicit compatibility path for callers that require + // the entire tmux scrollback. The browser uses `historyPage=1` for ordinary + // cold loads, while tail/no-param callers retain the bounded legacy path. + let tailBytes = query.tail ? parseInt(query.tail, 10) : 0; const isFullReload = query.full === '1' || query.full === 'true'; + const isLatestOnly = query.latest === '1' || query.latest === 'true'; + const isHistoryPage = query.historyPage === '1' || query.historyPage === 'true'; + const captureFullHistory = isFullReload && !isLatestOnly; const { tmuxHistoryLimit, terminalBufferMaxBytes } = await ctx.getTerminalHistoryConfig(); + if (isHistoryPage && session.muxName && typeof ctx.mux.captureActivePaneHistoryPage === 'function') { + const parsedLines = Number.parseInt(query.lines || '', 10); + const limit = Number.isSafeInteger(parsedLines) ? Math.max(100, Math.min(2000, parsedLines)) : 1000; + const parsedBefore = query.before === undefined ? undefined : Number.parseInt(query.before, 10); + const parsedAfter = query.after === undefined ? undefined : Number.parseInt(query.after, 10); + const before = Number.isSafeInteger(parsedBefore) ? parsedBefore : undefined; + const after = before === undefined && Number.isSafeInteger(parsedAfter) ? parsedAfter : undefined; + const page = await ctx.mux.captureActivePaneHistoryPage(session.muxName, { + limit, + before, + after, + }); + + if (page) { + const cursor = session.terminalCursor; + if (query.format === 'stream') { + reply + .type('text/plain; charset=utf-8') + .header('X-Codeman-Terminal-Format', 'stream-v1') + .header('Cache-Control', 'no-store') + .header('X-Codeman-Terminal-Stream', cursor.stream) + .header('X-Codeman-Terminal-Generation', String(cursor.generation)) + .header('X-Codeman-Terminal-Start', String(cursor.start)) + .header('X-Codeman-Terminal-End', String(cursor.end)) + .header('X-Codeman-Terminal-Status', session.status) + .header('X-Codeman-Terminal-Full-Size', String(page.buffer.length)) + .header('X-Codeman-Terminal-Truncated', '0') + .header('X-Codeman-Terminal-Source', 'mux-history-page') + .header('X-Codeman-History-Start', String(page.start)) + .header('X-Codeman-History-End', String(page.end)) + .header('X-Codeman-History-Total', String(page.total)) + .header('X-Codeman-History-More-Before', page.hasMoreBefore ? '1' : '0') + .header('X-Codeman-History-More-After', page.hasMoreAfter ? '1' : '0') + .header('X-Codeman-History-Origin', page.origin); + return reply.send(createTerminalSnapshotStream(page.buffer)); + } + + return { + terminalBuffer: page.buffer, + status: session.status, + fullSize: page.buffer.length, + truncated: false, + source: 'mux-history-page', + cursor, + historyPage: { + start: page.start, + end: page.end, + total: page.total, + hasMoreBefore: page.hasMoreBefore, + hasMoreAfter: page.hasMoreAfter, + origin: page.origin, + }, + }; + } + } + // A transient tmux capture failure must remain bounded. Falling through + // with tailBytes=0 would return the entire in-memory PTY buffer and undo + // the bandwidth guarantee of the paging request. + if (isHistoryPage && tailBytes <= 0) { + tailBytes = TERMINAL_HISTORY_PAGE_FALLBACK_BYTES; + } + // Prepend the live tmux pane buffer so tab-switch replay shows the current // on-screen frame, not just the accumulated byte history. This matters for // TUI modes (codex/opencode) that repaint only their latest frame: the @@ -1597,14 +1692,14 @@ export function registerSessionRoutes( muxName && typeof ctx.mux.captureActivePaneBuffer === 'function' ? ctx.mux.captureActivePaneBuffer( muxName, - isFullReload + captureFullHistory ? { fullHistory: true, historyLimitLines: tmuxHistoryLimit, maxCaptureBytes: terminalBufferMaxBytes } : undefined ) : null; const hasLiveMuxBuffer = liveMuxBuffer !== null && liveMuxBuffer.length > 0; const source: 'history' | 'mux-visible' | 'mux-full-history' = hasLiveMuxBuffer - ? isFullReload + ? captureFullHistory ? 'mux-full-history' : 'mux-visible' : 'history'; @@ -1615,11 +1710,13 @@ export function registerSessionRoutes( // history would replay the whole conversation twice: `\x1b[2J` clears only // the viewport, not xterm scrollback. The history+clear+frame concat stays // for the visible-frame path, where the single pane frame lacks history. - rawBuffer = isFullReload + rawBuffer = isLatestOnly ? liveMuxBuffer - : session.terminalBufferLength > 0 - ? `${session.terminalBuffer}\x1b[H\x1b[2J${liveMuxBuffer}` - : liveMuxBuffer; + : captureFullHistory + ? liveMuxBuffer + : session.terminalBufferLength > 0 + ? `${session.terminalBuffer}\x1b[H\x1b[2J${liveMuxBuffer}` + : liveMuxBuffer; } else { rawBuffer = session.terminalBuffer; } @@ -1688,12 +1785,30 @@ export function registerSessionRoutes( // Remove Ctrl+L and leading whitespace (cheap on tailed subset) cleanBuffer = cleanBuffer.replace(CTRL_L_PATTERN, '').replace(LEADING_WHITESPACE_PATTERN, ''); + const cursor = session.terminalCursor; + if (query.format === 'stream') { + reply + .type('text/plain; charset=utf-8') + .header('X-Codeman-Terminal-Format', 'stream-v1') + .header('Cache-Control', 'no-store') + .header('X-Codeman-Terminal-Stream', cursor.stream) + .header('X-Codeman-Terminal-Generation', String(cursor.generation)) + .header('X-Codeman-Terminal-Start', String(cursor.start)) + .header('X-Codeman-Terminal-End', String(cursor.end)) + .header('X-Codeman-Terminal-Status', session.status) + .header('X-Codeman-Terminal-Full-Size', String(fullSize)) + .header('X-Codeman-Terminal-Truncated', truncated ? '1' : '0') + .header('X-Codeman-Terminal-Source', source); + return reply.send(createTerminalSnapshotStream(cleanBuffer.replace(DEC_SYNC_OUTPUT_PATTERN, ''))); + } + return { terminalBuffer: cleanBuffer, status: session.status, fullSize, truncated, source, + cursor, }; }); diff --git a/src/web/routes/ws-routes.ts b/src/web/routes/ws-routes.ts index 1b59d6eb..52f65f3c 100644 --- a/src/web/routes/ws-routes.ts +++ b/src/web/routes/ws-routes.ts @@ -11,26 +11,32 @@ * paths remain fully functional. The frontend opts into WS when available and * falls back transparently. * - * Terminal output is micro-batched at 8ms to group Ink's rapid cursor-up redraws - * into single frames, preventing flicker from split ANSI sequences. This matches - * the SSE path's server-side batching (16-50ms) but at a shorter interval since - * WS has no Traefik buffering overhead. + * Terminal output is micro-batched to group Ink's rapid cursor-up redraws into + * single frames, preventing flicker from split ANSI sequences. Desktop keeps an + * 8ms/8KB low-latency budget. Mobile uses 50ms/16KB because synchronized xterm + * paints are substantially more expensive on phone CPUs; grouping before the + * DEC-2026 wrapper preserves every PTY byte while removing redundant paints. * * Protocol (all JSON text frames): * Server -> Client: - * {"t":"o","d":"..."} — terminal output + * {"t":"o","d":"...","cursor":{...}} — terminal output with an optional stream cursor * {"t":"c"} — clear terminal * {"t":"r"} — needs refresh (reload buffer) * {"t":"ia","seq":N} — input ACK (echoes the seq of an applied/deduped input frame) + * {"t":"ha"} — terminal-stream handoff ACK * Client -> Server: * {"t":"i","d":"...","seq":N,"cid":"..."} — input (keystroke or paste). seq+cid are * optional reliable-delivery tags: the server applies each * (cid,seq) at-most-once and ACKs with {"t":"ia","seq":N}. - * {"t":"z","c":N,"r":N,"f":bool} — resize terminal (f=true forces SIGWINCH even if dims unchanged) + * Input also reasserts the connection's last announced viewport. + * {"t":"z","c":N,"r":N,"v":"mobile|tablet|desktop","f":bool} + * — resize terminal. f forces SIGWINCH. + * {"t":"h"} — flush WS output and atomically hand terminal delivery to SSE */ import { FastifyInstance } from 'fastify'; import type { WebSocket } from 'ws'; +import type { TerminalCursor } from '../../session.js'; import type { SessionPort } from '../ports/session-port.js'; import { MAX_INPUT_LENGTH } from '../../config/terminal-limits.js'; import { isAllowedRequestHost, isAllowedRequestOrigin, type HostPolicy } from '../network-auth-policy.js'; @@ -41,8 +47,17 @@ import { canAccessOwned, getAuthUser } from '../route-helpers.js'; * long enough to group Ink's rapid cursor-up redraw sequences into single frames. */ const WS_BATCH_INTERVAL_MS = 8; -/** Flush immediately when batch exceeds this size (bytes) for responsiveness. */ -const WS_BATCH_FLUSH_THRESHOLD = 16384; +/** Keep active-view redraw frames small enough for one smooth mobile paint. */ +const WS_BATCH_FLUSH_THRESHOLD = 8 * 1024; + +/** Phone-specific batching budget: reliably groups two ~30Hz source animation ticks. */ +const WS_MOBILE_BATCH_INTERVAL_MS = 50; + +/** Match the browser's mobile xterm parse budget while retaining progressive bulk output. */ +const WS_MOBILE_BATCH_FLUSH_THRESHOLD = 16 * 1024; + +/** Bound one phone render transaction without defeating the 50ms redraw grouping window. */ +const WS_MOBILE_BATCH_HARD_LIMIT = 256 * 1024; /** How often to ping each WebSocket client (ms). Detects stale connections that * TCP keepalive won't catch for minutes, especially through tunnels/proxies. */ @@ -61,6 +76,143 @@ const DEC_2026_END = '\x1b[?2026l'; /** Max concurrent WS connections per session. Prevents listener/bandwidth multiplication. */ const MAX_WS_PER_SESSION = 5; +function splitTerminalPayload( + data: string, + maxBytes: number, + flushSafeTail: boolean +): { frames: string[]; remainder: string } { + if (!data) return { frames: [], remainder: '' }; + const chunks: string[] = []; + let parserState: 'ground' | 'escape' | 'csi' | 'osc' | 'osc-escape' | 'control-string' | 'control-string-escape' = + 'ground'; + let controlSequence = ''; + let synchronizedUpdateOpen = false; + let start = 0; + let offset = 0; + let chunkBytes = 0; + let lastSafeOffset = 0; + while (offset < data.length) { + const codePoint = data.codePointAt(offset) ?? 0; + const codeUnits = codePoint > 0xffff ? 2 : 1; + const codePointBytes = codePoint <= 0x7f ? 1 : codePoint <= 0x7ff ? 2 : codePoint <= 0xffff ? 3 : 4; + + // Split before the next ground-state code point when it would exceed the + // target. Inside ANSI strings/sequences, correctness wins: wait until the + // parser returns to ground rather than injecting a sync marker mid-command. + if ( + parserState === 'ground' && + !synchronizedUpdateOpen && + chunkBytes > 0 && + chunkBytes + codePointBytes > maxBytes + ) { + chunks.push(data.slice(start, offset)); + start = offset; + chunkBytes = 0; + lastSafeOffset = start; + } + + chunkBytes += codePointBytes; + offset += codeUnits; + + switch (parserState) { + case 'ground': + if (codePoint === 0x1b) { + parserState = 'escape'; + controlSequence = '\x1b'; + } else if (codePoint === 0x9b) { + parserState = 'csi'; + controlSequence = '\x9b'; + } else if (codePoint === 0x9d) { + parserState = 'osc'; + } else if (codePoint === 0x90 || codePoint === 0x98 || codePoint === 0x9e || codePoint === 0x9f) { + parserState = 'control-string'; + } + break; + case 'escape': + controlSequence += String.fromCodePoint(codePoint); + if (codePoint === 0x5b) { + parserState = 'csi'; + } else if (codePoint === 0x5d) { + parserState = 'osc'; + controlSequence = ''; + } else if (codePoint === 0x50 || codePoint === 0x58 || codePoint === 0x5e || codePoint === 0x5f) { + parserState = 'control-string'; + controlSequence = ''; + } else if (codePoint < 0x20 || codePoint > 0x2f) { + parserState = 'ground'; + controlSequence = ''; + } + break; + case 'csi': + if (codePoint === 0x1b) { + parserState = 'escape'; + controlSequence = '\x1b'; + } else if (codePoint === 0x18 || codePoint === 0x1a) { + parserState = 'ground'; + controlSequence = ''; + } else { + controlSequence += String.fromCodePoint(codePoint); + if (codePoint >= 0x40 && codePoint <= 0x7e) { + if (controlSequence === '\x1b[?2026h' || controlSequence === '\x9b?2026h') { + synchronizedUpdateOpen = true; + } else if (controlSequence === '\x1b[?2026l' || controlSequence === '\x9b?2026l') { + synchronizedUpdateOpen = false; + } + parserState = 'ground'; + controlSequence = ''; + } + } + break; + case 'osc': + if (codePoint === 0x07 || codePoint === 0x9c) { + parserState = 'ground'; + } else if (codePoint === 0x1b) { + parserState = 'osc-escape'; + } + break; + case 'osc-escape': + if (codePoint === 0x5c || codePoint === 0x9c || codePoint === 0x07) { + parserState = 'ground'; + } else { + parserState = codePoint === 0x1b ? 'osc-escape' : 'osc'; + } + break; + case 'control-string': + if (codePoint === 0x9c) { + parserState = 'ground'; + } else if (codePoint === 0x1b) { + parserState = 'control-string-escape'; + } + break; + case 'control-string-escape': + if (codePoint === 0x5c || codePoint === 0x9c) { + parserState = 'ground'; + } else { + parserState = codePoint === 0x1b ? 'control-string-escape' : 'control-string'; + } + break; + } + + if (parserState === 'ground' && !synchronizedUpdateOpen) { + lastSafeOffset = offset; + if (chunkBytes >= maxBytes) { + chunks.push(data.slice(start, offset)); + start = offset; + chunkBytes = 0; + lastSafeOffset = start; + } + } + } + if (flushSafeTail && lastSafeOffset > start) { + chunks.push(data.slice(start, lastSafeOffset)); + start = lastSafeOffset; + } + return { + frames: chunks, + remainder: data.slice(start), + }; +} + /** * Track live WS connections per session, keyed by clientId (COD-137). * Replaces a bare counter that over-counted across the async-close gap on @@ -70,7 +222,22 @@ const MAX_WS_PER_SESSION = 5; */ const sessionWsRegistry = new WsConnectionRegistry(MAX_WS_PER_SESSION); -export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHostPolicy: () => HostPolicy): void { +export interface TerminalStreamCoordinator { + suspendSseTerminal(clientId: string, sessionId: string): void; + resumeSseTerminal(clientId: string, sessionId: string, recover: boolean): void; +} + +const NOOP_STREAM_COORDINATOR: TerminalStreamCoordinator = { + suspendSseTerminal: () => {}, + resumeSseTerminal: () => {}, +}; + +export function registerWsRoutes( + app: FastifyInstance, + ctx: SessionPort, + getHostPolicy: () => HostPolicy, + streamCoordinator: TerminalStreamCoordinator = NOOP_STREAM_COORDINATOR +): void { app.get<{ Params: { id: string }; Querystring: { cid?: string } }>( '/ws/sessions/:id/terminal', { websocket: true }, @@ -134,29 +301,95 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost } console.info('[ws] terminal open', { sessionId: id, wsCount: sessionWsRegistry.liveCount(id) }); + if (cid) { + streamCoordinator.suspendSseTerminal(cid, id); + } + + let transportReleased = false; + let detachTerminalTransport = () => {}; + const releaseTerminalTransport = (recover: boolean) => { + if (transportReleased || !cid) return; + transportReleased = true; + if (sessionWsRegistry.hasSocket(id, socket)) { + streamCoordinator.resumeSseTerminal(cid, id, recover); + } + }; + // Eagerly free the slot on error/terminate — don't wait for the async // 'close' (idempotent with the 'close' handler below). This is what kills // the reconnect over-count: the slot is released the instant the socket dies. socket.on('error', () => { + releaseTerminalTransport(true); sessionWsRegistry.unregister(id, socket); }); // Per-connection micro-batch state let batchChunks: string[] = []; let batchSize = 0; + let batchCursor: TerminalCursor | undefined; let batchTimer: ReturnType | null = null; + let announcedViewport: 'mobile' | 'tablet' | 'desktop' | undefined; + const batchIntervalMs = () => + announcedViewport === 'mobile' ? WS_MOBILE_BATCH_INTERVAL_MS : WS_BATCH_INTERVAL_MS; + const batchFlushThreshold = () => + announcedViewport === 'mobile' ? WS_MOBILE_BATCH_FLUSH_THRESHOLD : WS_BATCH_FLUSH_THRESHOLD; + const resetBatch = () => { + batchChunks = []; + batchSize = 0; + batchCursor = undefined; + }; - const flushBatch = () => { + const flushBatch = (flushSafeTail = true) => { batchTimer = null; if (batchChunks.length === 0 || socket.readyState !== 1) { - batchChunks = []; - batchSize = 0; + resetBatch(); return; } const data = batchChunks.join(''); - batchChunks = []; - batchSize = 0; - socket.send(`{"t":"o","d":${JSON.stringify(DEC_2026_START + data + DEC_2026_END)}}`); + const cursor = batchCursor; + const { frames, remainder } = splitTerminalPayload(data, batchFlushThreshold(), flushSafeTail); + batchChunks = remainder ? [remainder] : []; + batchSize = Buffer.byteLength(remainder, 'utf8'); + let cursorOffset = 0; + for (let index = 0; index < frames.length; index += 1) { + // Transport fragments from one flush are one render transaction. Keep + // the DEC synchronized update open across WS messages so xterm can + // parse bounded chunks without exposing each partial Codex redraw. + const synchronizedFrame = + (index === 0 ? DEC_2026_START : '') + frames[index] + (index === frames.length - 1 ? DEC_2026_END : ''); + const frameCursor = cursor + ? { + ...cursor, + start: cursor.start + cursorOffset, + end: cursor.start + cursorOffset + frames[index].length, + } + : undefined; + socket.send( + JSON.stringify({ + t: 'o', + d: synchronizedFrame, + ...(frameCursor ? { cursor: frameCursor } : {}), + }) + ); + cursorOffset += frames[index].length; + } + batchCursor = + cursor && remainder + ? { + ...cursor, + start: cursor.start + cursorOffset, + } + : undefined; + }; + const flushBatchNow = () => { + if (batchTimer) { + clearTimeout(batchTimer); + batchTimer = null; + } + // A threshold flush is still one logical PTY emission. Send its safe + // tail in the same synchronized transaction; splitTerminalPayload keeps + // only a genuinely incomplete ANSI/control sequence for the next event. + flushBatch(); }; // Per-connection desktop sizing claim — registered on the first @@ -189,6 +422,17 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost if (seq !== null && socket.readyState === 1) { socket.send(`{"t":"ia","seq":${seq}}`); } + } else if (msg.t === 'h') { + if (batchTimer) { + clearTimeout(batchTimer); + batchTimer = null; + } + flushBatch(); + const needsRecovery = batchChunks.length > 0; + resetBatch(); + detachTerminalTransport(); + releaseTerminalTransport(needsRecovery); + if (socket.readyState === 1) socket.send('{"t":"ha"}'); } else if ( msg.t === 'z' && Number.isInteger(msg.c) && @@ -199,6 +443,7 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost msg.r <= 200 ) { const viewportType = msg.v === 'mobile' || msg.v === 'tablet' || msg.v === 'desktop' ? msg.v : undefined; + announcedViewport = viewportType; if (viewportType === 'desktop') { session.claimDesktopSizing(sizingToken); holdsDesktopClaim = true; @@ -217,27 +462,49 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost }); // Terminal output -> micro-batched WS send - const onTerminal = (data: string) => { + const onTerminal = (data: string, cursor?: TerminalCursor) => { if (socket.readyState !== 1) return; - batchChunks.push(data); - batchSize += data.length; - - // Flush immediately for large batches (responsiveness during bulk output) - if (batchSize > WS_BATCH_FLUSH_THRESHOLD) { - if (batchTimer) { - clearTimeout(batchTimer); + const cursorIsContiguous = + cursor && + batchCursor && + cursor.stream === batchCursor.stream && + cursor.generation === batchCursor.generation && + cursor.start === batchCursor.end; + if ( + batchChunks.length > 0 && + ((cursor && !batchCursor) || (!cursor && batchCursor) || (cursor && batchCursor && !cursorIsContiguous)) + ) { + flushBatchNow(); + if (batchChunks.length > 0) { + // A partial control sequence cannot cross a cursor discontinuity. + // Drop only that unsafe suffix and force an authoritative reload. + resetBatch(); + socket.send('{"t":"r"}'); } - flushBatch(); - return; } + batchChunks.push(data); + batchSize += Buffer.byteLength(data, 'utf8'); + if (cursor) { + batchCursor = batchCursor ? { ...batchCursor, end: cursor.end } : { ...cursor }; + } + const immediateFlushLimit = announcedViewport === 'mobile' ? WS_MOBILE_BATCH_HARD_LIMIT : batchFlushThreshold(); + if (batchSize >= immediateFlushLimit) flushBatchNow(); // Start timer if not already running - if (!batchTimer) { - batchTimer = setTimeout(flushBatch, WS_BATCH_INTERVAL_MS); + if (batchChunks.length > 0 && !batchTimer) { + batchTimer = setTimeout(flushBatch, batchIntervalMs()); } }; const onClearTerminal = () => { + if (batchTimer) { + clearTimeout(batchTimer); + batchTimer = null; + } + flushBatch(); + // A clear starts a new terminal generation. Any incomplete escape + // suffix from the old generation is no longer meaningful. + resetBatch(); if (socket.readyState === 1) { socket.send('{"t":"c"}'); } @@ -259,6 +526,11 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost session.on('clearTerminal', onClearTerminal); session.on('needsRefresh', onNeedsRefresh); session.on('exit', onSessionExit); + detachTerminalTransport = () => { + session.off('terminal', onTerminal); + session.off('clearTerminal', onClearTerminal); + session.off('needsRefresh', onNeedsRefresh); + }; // Heartbeat: detect stale connections (especially through tunnels where // TCP RST can take minutes to propagate). @@ -278,6 +550,7 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost console.warn('[ws] terminal ping timeout — terminating', { sessionId: id }); // Free the slot eagerly — terminate()'s 'close' may lag, and a client // reconnecting after a stale-connection drop must not be over-counted. + releaseTerminalTransport(true); sessionWsRegistry.unregister(id, socket); socket.terminate(); }, WS_PONG_TIMEOUT_MS); @@ -287,13 +560,12 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost clearInterval(pingInterval); if (pongTimeout) clearTimeout(pongTimeout); if (batchTimer) clearTimeout(batchTimer); - batchChunks = []; - session.off('terminal', onTerminal); - session.off('clearTerminal', onClearTerminal); - session.off('needsRefresh', onNeedsRefresh); + resetBatch(); + detachTerminalTransport(); session.off('exit', onSessionExit); session.releaseDesktopSizing(sizingToken); + releaseTerminalTransport(code !== 4009); // Release this socket's slot. Idempotent and identity-matched: if this // socket was already superseded (a same-cid reconnect took its slot) or // eagerly unregistered on terminate/error, this is a no-op and the diff --git a/src/web/server.ts b/src/web/server.ts index ecbc71d0..aad64077 100644 --- a/src/web/server.ts +++ b/src/web/server.ts @@ -43,7 +43,7 @@ import { hostname as getHostname } from 'node:os'; import { dataPath, getDataDir, CODEMAN_INSTANCE } from '../config/instance.js'; import { getHookSecret } from '../config/hook-secret.js'; import { EventEmitter } from 'node:events'; -import { Session, isExternalCliMode, type BackgroundTask } from '../session.js'; +import { Session, isExternalCliMode, type BackgroundTask, type TerminalCursor } from '../session.js'; import type { ClaudeMode, SessionAttachmentHistoryItem, SessionState, WorkflowRunInfo } from '../types.js'; import { RespawnController, RespawnConfig } from '../respawn-controller.js'; import type { TerminalMultiplexer } from '../mux-interface.js'; @@ -167,10 +167,10 @@ import { CronService } from '../cron/cron-service.js'; const __dirname = dirname(fileURLToPath(import.meta.url)); -// Bounded, predictable shape for SSE client identifiers: alphanumerics, `_`, `-`. -// Length range covers crypto.randomUUID() (36 chars) plus any short stable IDs, -// while capping growth of `sseClientsById` and blocking pathological inputs. -const SSE_CLIENT_ID_RE = /^[A-Za-z0-9_-]{8,64}$/; +// Bounded, predictable shape for SSE client identifiers: alphanumerics, `_`, `-`, +// and `:` between the stable browser id and per-tab nonce. The length cap bounds +// growth of `sseClientsById` and rejects pathological inputs. +const SSE_CLIENT_ID_RE = /^[A-Za-z0-9_:-]{8,128}$/; function escapeHtmlText(value: string): string { return value.replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>'); @@ -793,14 +793,12 @@ export class WebServer extends EventEmitter { // POST /api/events/subscribe without reconnecting. const query = req.query as { sessions?: string; clientId?: string }; let sessionFilter: Set | null = null; - if (query.sessions) { + if (query.sessions !== undefined) { const ids = query.sessions .split(',') .map((s) => s.trim()) .filter(Boolean); - if (ids.length > 0) { - sessionFilter = new Set(ids); - } + sessionFilter = new Set(ids); } const clientId = typeof query.clientId === 'string' && SSE_CLIENT_ID_RE.test(query.clientId) ? query.clientId : undefined; @@ -832,7 +830,7 @@ export class WebServer extends EventEmitter { // Live subscription update — change a connected client's session filter // without forcing an SSE reconnect. Body: { clientId, sessions: string[] | null } - // Empty/null sessions array = remove filter (receive all session:terminal events). + // Empty array = receive no terminal events; null = legacy unfiltered mode. this.app.post('/api/events/subscribe', (req, reply) => { const body = (req.body || {}) as { clientId?: string; sessions?: string[] | null }; if (typeof body.clientId !== 'string' || !SSE_CLIENT_ID_RE.test(body.clientId)) { @@ -939,7 +937,14 @@ export class WebServer extends EventEmitter { this.cronService.init(); registerCronRoutes(this.app, { ...ctx, cron: this.cronService }); - registerWsRoutes(this.app, ctx, () => this.getHostPolicy()); + registerWsRoutes(this.app, ctx, () => this.getHostPolicy(), { + suspendSseTerminal: (clientId, sessionId) => { + this.sse.suspendClientSession(clientId, sessionId); + }, + resumeSseTerminal: (clientId, sessionId, recover) => { + this.sse.resumeClientSession(clientId, sessionId, recover); + }, + }); } /** @@ -1965,8 +1970,8 @@ export class WebServer extends EventEmitter { return undefined; } - private batchTerminalData(sessionId: string, data: string): void { - this.sse.batchTerminalData(sessionId, data); + private batchTerminalData(sessionId: string, data: string, cursor?: TerminalCursor): void { + this.sse.batchTerminalData(sessionId, data, cursor); } private batchTaskUpdate(sessionId: string, task: BackgroundTask): void { diff --git a/src/web/session-listener-wiring.ts b/src/web/session-listener-wiring.ts index 3f040e7d..7c225fd3 100644 --- a/src/web/session-listener-wiring.ts +++ b/src/web/session-listener-wiring.ts @@ -22,6 +22,7 @@ import type { RalphTrackerState, RalphTodoItem, ActiveBashTool, + TerminalCursor, } from '../session.js'; import type { RalphStatusBlock, CircuitBreakerStatus } from '../types.js'; import { SseEvent } from './sse-events.js'; @@ -30,7 +31,7 @@ import { fileStreamManager } from '../file-stream-manager.js'; /** Stored listener references for session cleanup (prevents memory leaks) */ export interface SessionListenerRefs { - terminal: (data: string) => void; + terminal: (data: string, cursor?: TerminalCursor) => void; clearTerminal: () => void; needsRefresh: () => void; message: (msg: ClaudeMessage) => void; @@ -65,7 +66,7 @@ export interface SessionListenerRefs { /** Dependencies injected by WebServer — keeps listener creation decoupled from server internals. */ interface SessionListenerDeps { broadcast(event: string, data: unknown): void; - batchTerminalData(sessionId: string, data: string): void; + batchTerminalData(sessionId: string, data: string, cursor?: TerminalCursor): void; batchTaskUpdate(sessionId: string, task: BackgroundTask): void; broadcastSessionStateDebounced(sessionId: string): void; sendPushNotifications(event: string, data: Record): void; @@ -91,8 +92,8 @@ export function createSessionListeners(session: Session, deps: SessionListenerDe // ─── Terminal Output ───────────────────────────────────── /** Batches PTY output → broadcasts `session:terminal` at 16-50ms intervals */ - terminal: (data) => { - deps.batchTerminalData(session.id, data); + terminal: (data, cursor) => { + deps.batchTerminalData(session.id, data, cursor); }, /** Broadcasts `session:clearTerminal` — tells clients to wipe their xterm buffer (after mux attach) */ diff --git a/src/web/sse-stream-manager.ts b/src/web/sse-stream-manager.ts index 32c65f71..9faab384 100644 --- a/src/web/sse-stream-manager.ts +++ b/src/web/sse-stream-manager.ts @@ -16,7 +16,7 @@ */ import type { FastifyReply } from 'fastify'; -import type { BackgroundTask } from '../session.js'; +import type { BackgroundTask, TerminalCursor } from '../session.js'; import type { AuthUser } from '../types.js'; import { CleanupManager, StaleExpirationMap } from '../utils/index.js'; import { SseEvent } from './sse-events.js'; @@ -71,6 +71,10 @@ export class SseStreamManager { private sseClients: Map | null> = new Map(); /** Optional client-supplied IDs → reply, for live filter updates without reconnecting */ private sseClientsById: Map = new Map(); + /** Reverse lookup used by per-client terminal transport suspension. */ + private sseClientIds: Map = new Map(); + /** Sessions temporarily owned by a client's WebSocket transport. */ + private suspendedClientSessions: Map> = new Map(); /** Per-client identity (multi-user); absent for single-user clients → no filtering. */ private sseClientIdentity: Map = new Map(); /** SSE clients connecting from non-localhost (i.e. through tunnel) */ @@ -85,6 +89,7 @@ export class SseStreamManager { // ─── Terminal Batching ────────────────────────────────── private terminalBatches: Map = new Map(); private terminalBatchSizes: Map = new Map(); // Running total avoids O(n) reduce per push + private terminalBatchCursors: Map = new Map(); private terminalBatchTimers: Map = new Map(); // Per-session timers (staggered flushes) // Adaptive batching: track rapid events to extend batch window (per-session) // StaleExpirationMap auto-cleans entries for sessions that stop generating output @@ -148,8 +153,10 @@ export class SseStreamManager { this.remoteSseClients.delete(prev); this.backpressuredClients.delete(prev); this.sseClientIdentity.delete(prev); + this.sseClientIds.delete(prev); } this.sseClientsById.set(clientId, reply); + this.sseClientIds.set(reply, clientId); } } @@ -158,6 +165,7 @@ export class SseStreamManager { this.remoteSseClients.delete(reply); this.backpressuredClients.delete(reply); this.sseClientIdentity.delete(reply); + this.sseClientIds.delete(reply); // Clear any clientId mappings pointing at this reply for (const [id, r] of this.sseClientsById) { if (r === reply) this.sseClientsById.delete(id); @@ -189,11 +197,44 @@ export class SseStreamManager { updateClientFilter(clientId: string, sessions: string[] | null): boolean { const reply = this.sseClientsById.get(clientId); if (!reply || !this.sseClients.has(reply)) return false; - const filter = sessions && sessions.length > 0 ? new Set(sessions) : null; + const filter = sessions === null ? null : new Set(sessions); this.sseClients.set(reply, filter); return true; } + /** + * Transfer one client's session stream from SSE to WebSocket without a gap. + * Any batch created before the WS listener attached is delivered first. + */ + suspendClientSession(clientId: string, sessionId: string): boolean { + this.flushSessionTerminalBatch(sessionId); + let sessions = this.suspendedClientSessions.get(clientId); + if (!sessions) { + sessions = new Set(); + this.suspendedClientSessions.set(clientId, sessions); + } + sessions.add(sessionId); + return this.sseClientsById.has(clientId); + } + + /** + * Transfer a session back to SSE. Flush while it is still suppressed so bytes + * already delivered over WS cannot be replayed from an in-flight SSE batch. + */ + resumeClientSession(clientId: string, sessionId: string, recover: boolean): boolean { + this.flushSessionTerminalBatch(sessionId); + const sessions = this.suspendedClientSessions.get(clientId); + sessions?.delete(sessionId); + if (sessions?.size === 0) this.suspendedClientSessions.delete(clientId); + + const reply = this.sseClientsById.get(clientId); + if (!reply || !this.sseClients.has(reply)) return false; + if (recover) { + this.sendSSE(reply, SseEvent.SessionNeedsRefresh, { id: sessionId }); + } + return true; + } + /** Send a single SSE event to a specific client. */ sendSSE(reply: FastifyReply, event: string, data: unknown): void { try { @@ -285,10 +326,25 @@ export class SseStreamManager { // Batch terminal data for better performance (60fps) // Uses per-session timers with adaptive intervals to prevent thundering herd: // each session flushes independently rather than all sessions flushing in one burst. - batchTerminalData(sessionId: string, data: string): void { + batchTerminalData(sessionId: string, data: string, cursor?: TerminalCursor): void { // Skip if server is stopping if (this._isStopping) return; + const currentSize = this.terminalBatchSizes.get(sessionId) ?? 0; + const currentCursor = this.terminalBatchCursors.get(sessionId); + const cursorIsContiguous = + cursor && + currentCursor && + cursor.stream === currentCursor.stream && + cursor.generation === currentCursor.generation && + cursor.start === currentCursor.end; + if ( + currentSize > 0 && + ((cursor && !currentCursor) || (!cursor && currentCursor) || (cursor && currentCursor && !cursorIsContiguous)) + ) { + this.flushSessionTerminalBatch(sessionId); + } + let chunks = this.terminalBatches.get(sessionId); if (!chunks) { chunks = []; @@ -298,6 +354,15 @@ export class SseStreamManager { const prevSize = this.terminalBatchSizes.get(sessionId) ?? 0; const totalLength = prevSize + data.length; this.terminalBatchSizes.set(sessionId, totalLength); + if (cursor) { + const batchCursor = this.terminalBatchCursors.get(sessionId); + this.terminalBatchCursors.set(sessionId, { + stream: cursor.stream, + generation: cursor.generation, + start: batchCursor?.start ?? cursor.start, + end: cursor.end, + }); + } // Adaptive batching: detect rapid events and extend batch window (per-session) const now = Date.now(); @@ -346,6 +411,7 @@ export class SseStreamManager { if (this._isStopping) { this.terminalBatches.delete(sessionId); this.terminalBatchSizes.delete(sessionId); + this.terminalBatchCursors.delete(sessionId); return; } const chunks = this.terminalBatches.get(sessionId); @@ -362,10 +428,13 @@ export class SseStreamManager { // Fast path: build SSE message directly without JSON.stringify on wrapper object. // Only the terminal data string needs escaping; sessionId is a UUID (safe to template). const escapedData = JSON.stringify(syncData); + const cursor = this.terminalBatchCursors.get(sessionId); + const cursorField = cursor ? `,"cursor":${JSON.stringify(cursor)}` : ''; // Append tunnel padding for immediate Cloudflare proxy flush — // terminal data is high-frequency and latency-sensitive. const padding = this._isTunnelActive ? SSE_PADDING : ''; - const message = `event: session:terminal\ndata: {"id":"${sessionId}","data":${escapedData}}\n\n` + padding; + const message = + `event: session:terminal\ndata: {"id":"${sessionId}","data":${escapedData}${cursorField}}\n\n` + padding; // Raw terminal bytes are the highest-value payload: resolve the session owner // ONCE and withhold the batch from any non-admin who is not the owner (fail // closed if the owner is unknown). No-op for identity-less single-user clients. @@ -374,12 +443,15 @@ export class SseStreamManager { for (const [client, filter] of this.sseClients) { // Skip clients that have a session filter and aren't subscribed to this session if (filter && !filter.has(sessionId)) continue; + const clientId = this.sseClientIds.get(client); + if (clientId && this.suspendedClientSessions.get(clientId)?.has(sessionId)) continue; if (!this.canDeliver(client, termHint)) continue; this.sendSSEPreformatted(client, message); } } this.terminalBatches.delete(sessionId); this.terminalBatchSizes.delete(sessionId); + this.terminalBatchCursors.delete(sessionId); } // ========== Task Update Batching ========== @@ -501,6 +573,11 @@ export class SseStreamManager { this.sseClients.delete(client); this.remoteSseClients.delete(client); this.backpressuredClients.delete(client); + const clientId = this.sseClientIds.get(client); + if (clientId && this.sseClientsById.get(clientId) === client) { + this.sseClientsById.delete(clientId); + } + this.sseClientIds.delete(client); } if (deadClients.length > 0) { @@ -514,6 +591,7 @@ export class SseStreamManager { cleanupSessionBatches(sessionId: string): void { this.terminalBatches.delete(sessionId); this.terminalBatchSizes.delete(sessionId); + this.terminalBatchCursors.delete(sessionId); const batchTimer = this.terminalBatchTimers.get(sessionId); if (batchTimer) { clearTimeout(batchTimer); @@ -545,6 +623,9 @@ export class SseStreamManager { } } this.sseClients.clear(); + this.sseClientsById.clear(); + this.sseClientIds.clear(); + this.suspendedClientSessions.clear(); this.remoteSseClients.clear(); this.backpressuredClients.clear(); @@ -555,6 +636,7 @@ export class SseStreamManager { this.terminalBatchTimers.clear(); this.terminalBatches.clear(); this.terminalBatchSizes.clear(); + this.terminalBatchCursors.clear(); this.taskUpdateBatches.clear(); this.stateUpdatePending.clear(); diff --git a/src/web/ws-connection-registry.ts b/src/web/ws-connection-registry.ts index f73d229c..b59e1866 100644 --- a/src/web/ws-connection-registry.ts +++ b/src/web/ws-connection-registry.ts @@ -103,17 +103,23 @@ export class WsConnectionRegistry e.socket === socket); - if (idx === -1) return; + if (idx === -1) return false; entries.splice(idx, 1); if (entries.length === 0) { this.bySession.delete(sessionId); } else { this.bySession.set(sessionId, entries); } + return true; + } + + /** True only while `socket` owns a live registry slot for the session. */ + hasSocket(sessionId: string, socket: S): boolean { + return this.bySession.get(sessionId)?.some((entry) => entry.socket === socket) ?? false; } /** Number of live entries for a session (0 if none). */ diff --git a/test/codex-terminal-output.test.ts b/test/codex-terminal-output.test.ts index 1e35ec1a..9a757d92 100644 --- a/test/codex-terminal-output.test.ts +++ b/test/codex-terminal-output.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest'; -import { Session } from '../src/session.js'; +import { Session, type TerminalCursor } from '../src/session.js'; type SessionInternals = { _handleTerminalOutput(data: string): void; @@ -11,6 +11,22 @@ function handleOutput(session: Session, data: string): void { } describe('Codex terminal output filtering', () => { + it('emits contiguous terminal cursor ranges and advances the generation after a clear', () => { + const session = new Session({ workingDir: '/tmp', mode: 'codex' }); + const cursors: TerminalCursor[] = []; + session.on('terminal', (_data, cursor: TerminalCursor) => cursors.push(cursor)); + + handleOutput(session, 'abc'); + handleOutput(session, 'de'); + session.clearBuffers(); + handleOutput(session, 'next'); + + expect(cursors[0]).toMatchObject({ generation: 0, start: 0, end: 3 }); + expect(cursors[1]).toMatchObject({ stream: cursors[0].stream, generation: 0, start: 3, end: 5 }); + expect(cursors[2]).toMatchObject({ stream: cursors[0].stream, generation: 1, start: 0, end: 4 }); + expect(session.terminalCursor).toEqual(cursors[2]); + }); + it('keeps browser scrollback guards but skips Codeman row repair in hybrid render mode', () => { const session = new Session({ workingDir: '/tmp', mode: 'codex', codexConfig: { renderMode: 'hybrid' } }); (session as unknown as SessionInternals)._ptyRows = 63; diff --git a/test/mocks/mock-route-context.ts b/test/mocks/mock-route-context.ts index 14aa1ea1..4359156c 100644 --- a/test/mocks/mock-route-context.ts +++ b/test/mocks/mock-route-context.ts @@ -106,6 +106,7 @@ export function createMockRouteContext(options?: { sessionId?: string }) { listSessions: vi.fn(() => []), getStats: vi.fn(() => ({})), updateSessionName: vi.fn(() => true), + updateSessionWorkingDir: vi.fn(() => true), getSession: vi.fn(() => null), clearRespawnConfig: vi.fn(), updateRespawnConfig: vi.fn(), diff --git a/test/mocks/mock-session.ts b/test/mocks/mock-session.ts index bda2077b..7249f1a7 100644 --- a/test/mocks/mock-session.ts +++ b/test/mocks/mock-session.ts @@ -4,6 +4,7 @@ */ import { EventEmitter } from 'node:events'; import { vi } from 'vitest'; +import type { TerminalCursor } from '../../src/session.js'; /** * Enhanced mock session for testing RespawnController. @@ -19,6 +20,7 @@ export class MockSession extends EventEmitter { ralphTracker: null = null; writeBuffer: string[] = []; terminalBuffer: string = ''; + private _terminalGeneration = 0; private _muxName: string | null = null; @@ -68,8 +70,13 @@ export class MockSession extends EventEmitter { /** Simulate raw terminal output */ simulateTerminalOutput(data: string): void { + const start = this.terminalBuffer.length; this.terminalBuffer += data; - this.emit('terminal', data); + this.emit('terminal', data, { + ...this.terminalCursor, + start, + end: this.terminalBuffer.length, + }); } /** Simulate prompt appearing (legacy fallback signal) */ @@ -152,6 +159,7 @@ export class MockSession extends EventEmitter { /** Clear terminal buffer */ clearTerminalBuffer(): void { this.terminalBuffer = ''; + this._terminalGeneration++; } // ========== Session Lifecycle ========== @@ -219,6 +227,15 @@ export class MockSession extends EventEmitter { return this.terminalBuffer.length; } + get terminalCursor(): TerminalCursor { + return { + stream: `mock-${this.id}`, + generation: this._terminalGeneration, + start: 0, + end: this.terminalBuffer.length, + }; + } + /** Return a state-like object for route handlers */ toState(): Record { return { diff --git a/test/routes/session-routes.test.ts b/test/routes/session-routes.test.ts index 259ed71d..e7d2677b 100644 --- a/test/routes/session-routes.test.ts +++ b/test/routes/session-routes.test.ts @@ -630,6 +630,117 @@ describe('session-routes', () => { expect(res.statusCode).toBe(200); const body = JSON.parse(res.body); expect(body.data.terminalBuffer).toBeDefined(); + expect(body.data.cursor).toEqual({ + stream: expect.any(String), + generation: expect.any(Number), + start: expect.any(Number), + end: 11, + }); + }); + + it('streams lossless terminal text outside the JSON envelope with cursor headers', async () => { + const terminalText = 'history line\nunicode: \u05e9\u05dc\u05d5\u05dd \ud83d\ude80\ncurrent prompt'; + harness.ctx._session.terminalBuffer = terminalText; + harness.ctx.mux.captureActivePaneBuffer = vi.fn(() => null); + + const res = await harness.app.inject({ + method: 'GET', + url: `/api/sessions/${harness.ctx._sessionId}/terminal?format=stream`, + }); + + expect(res.statusCode).toBe(200); + expect(res.body).toBe(terminalText); + expect(res.body).not.toContain('"success"'); + expect(res.headers['content-type']).toContain('text/plain'); + expect(res.headers['x-codeman-terminal-format']).toBe('stream-v1'); + expect(res.headers['cache-control']).toBe('no-store'); + expect(res.headers['x-codeman-terminal-stream']).toEqual(expect.any(String)); + expect(res.headers['x-codeman-terminal-generation']).toBe('0'); + expect(res.headers['x-codeman-terminal-start']).toBe('0'); + expect(res.headers['x-codeman-terminal-end']).toBe(String(terminalText.length)); + expect(res.headers['x-codeman-terminal-truncated']).toBe('0'); + expect(res.headers['x-codeman-terminal-source']).toBe('history'); + }); + + it('streams a bounded tmux history page with absolute row metadata', async () => { + const capturePage = vi.fn(() => ({ + buffer: 'PAGE_ROW_1200\r\nPAGE_ROW_2199', + start: 1200, + end: 2200, + total: 2200, + hasMoreBefore: true, + hasMoreAfter: false, + origin: 'pane-7:stable-origin', + })); + (harness.ctx.mux as { captureActivePaneHistoryPage?: unknown }).captureActivePaneHistoryPage = capturePage; + + const res = await harness.app.inject({ + method: 'GET', + url: `/api/sessions/${harness.ctx._sessionId}/terminal?historyPage=1&lines=1000&format=stream`, + }); + + expect(res.statusCode).toBe(200); + expect(res.body).toContain('PAGE_ROW_1200'); + expect(res.body).toContain('PAGE_ROW_2199'); + expect(res.headers['x-codeman-terminal-source']).toBe('mux-history-page'); + expect(res.headers['x-codeman-history-start']).toBe('1200'); + expect(res.headers['x-codeman-history-end']).toBe('2200'); + expect(res.headers['x-codeman-history-total']).toBe('2200'); + expect(res.headers['x-codeman-history-more-before']).toBe('1'); + expect(res.headers['x-codeman-history-more-after']).toBe('0'); + expect(res.headers['x-codeman-history-origin']).toBe('pane-7:stable-origin'); + expect(capturePage).toHaveBeenCalledWith(harness.ctx._session.muxName, { + limit: 1000, + before: undefined, + after: undefined, + }); + }); + + it('forwards an older-page boundary without requesting the full tmux history', async () => { + const capturePage = vi.fn(() => ({ + buffer: 'OLDER_PAGE', + start: 1000, + end: 2000, + total: 5000, + hasMoreBefore: true, + hasMoreAfter: true, + origin: 'pane-7:stable-origin', + })); + (harness.ctx.mux as { captureActivePaneHistoryPage?: unknown }).captureActivePaneHistoryPage = capturePage; + + const res = await harness.app.inject({ + method: 'GET', + url: `/api/sessions/${harness.ctx._sessionId}/terminal?historyPage=1&before=2000&lines=1000&format=stream`, + }); + + expect(res.statusCode).toBe(200); + expect(capturePage).toHaveBeenCalledWith(harness.ctx._session.muxName, { + limit: 1000, + before: 2000, + after: undefined, + }); + expect(harness.ctx.mux.captureActivePaneBuffer).toBeUndefined(); + }); + + it('falls back to a bounded newest-byte tail when tmux page capture fails', async () => { + const oldest = 'HISTORY_PAGE_FALLBACK_OLDEST'; + const newest = 'HISTORY_PAGE_FALLBACK_NEWEST'; + harness.ctx._session.terminalBuffer = oldest + '\n' + 'x'.repeat(1024 * 1024 + 4096) + '\n' + newest; + (harness.ctx.mux as { captureActivePaneHistoryPage?: unknown }).captureActivePaneHistoryPage = vi.fn(() => null); + (harness.ctx.mux as { captureActivePaneBuffer?: unknown }).captureActivePaneBuffer = vi.fn(() => null); + + const res = await harness.app.inject({ + method: 'GET', + url: `/api/sessions/${harness.ctx._sessionId}/terminal?historyPage=1`, + }); + + expect(res.statusCode).toBe(200); + const body = JSON.parse(res.body); + expect(body.data.terminalBuffer.length).toBeLessThanOrEqual(1024 * 1024); + expect(body.data.terminalBuffer).toContain(newest); + expect(body.data.terminalBuffer).not.toContain(oldest); + expect(body.data.truncated).toBe(true); + expect(body.data.historyPage).toBeUndefined(); }); it('does not strip VPA-like shell scrollback as Ink redraw bloat', async () => { @@ -676,6 +787,26 @@ describe('session-routes', () => { expect(harness.ctx.mux.captureActivePaneBuffer).toHaveBeenCalledWith(harness.ctx._session.muxName, undefined); }); + it('returns only the current mux pane for latest-frame requests, even with full=1', async () => { + harness.ctx._session.terminalBuffer = 'old accumulated history\nthat must load in the background'; + harness.ctx._session.mode = 'codex'; + (harness.ctx.mux as { captureActivePaneBuffer?: unknown }).captureActivePaneBuffer = vi.fn( + () => 'current pane now\n› current prompt' + ); + + const res = await harness.app.inject({ + method: 'GET', + url: `/api/sessions/${harness.ctx._sessionId}/terminal?latest=1&full=1&tail=131072&format=stream`, + }); + + expect(res.statusCode).toBe(200); + expect(res.body).toContain('current pane now'); + expect(res.body).toContain('› current prompt'); + expect(res.body).not.toContain('old accumulated history'); + expect(res.headers['x-codeman-terminal-source']).toBe('mux-visible'); + expect(harness.ctx.mux.captureActivePaneBuffer).toHaveBeenCalledWith(harness.ctx._session.muxName, undefined); + }); + // ── COD-47: full tmux scrollback replay on full page reload ── it('full reload (?full=1) requests full tmux history and replays boundary markers', async () => { // A realistic scrollback-length capture: ~5000 lines, well past one screen. diff --git a/test/routes/ws-routes.test.ts b/test/routes/ws-routes.test.ts index 5b722036..9077a36c 100644 --- a/test/routes/ws-routes.test.ts +++ b/test/routes/ws-routes.test.ts @@ -19,6 +19,21 @@ import { registerWsRoutes } from '../../src/web/routes/ws-routes.js'; import { MAX_INPUT_LENGTH } from '../../src/config/terminal-limits.js'; const PORT = 3170; +const DEC_2026_START = '\x1b[?2026h'; +const DEC_2026_END = '\x1b[?2026l'; + +function synchronizedPayload(messages: Array<{ d: string }>): string { + const transaction = messages.map((message) => message.d).join(''); + expect(transaction.startsWith(DEC_2026_START)).toBe(true); + expect(transaction.endsWith(DEC_2026_END)).toBe(true); + return transaction.slice(DEC_2026_START.length, -DEC_2026_END.length); +} + +function transportPayload(message: { d: string }, index: number, count: number): string { + const start = index === 0 ? DEC_2026_START.length : 0; + const end = index === count - 1 ? message.d.length - DEC_2026_END.length : message.d.length; + return message.d.slice(start, end); +} /** Helper: open a WebSocket connection and wait for it to reach OPEN state. */ function connectWs(path: string, timeoutMs = 5000): Promise { @@ -78,13 +93,26 @@ function collectMessages(ws: WebSocket, count: number, timeoutMs = 3000): Promis describe('ws-routes', () => { let app: FastifyInstance; let ctx: MockRouteContext; + let streamCoordinator: { + suspendSseTerminal: ReturnType; + resumeSseTerminal: ReturnType; + }; beforeEach(async () => { app = Fastify({ logger: false }); await app.register(fastifyWebsocket); ctx = createMockRouteContext({ sessionId: 'ws-test-session' }); - registerWsRoutes(app, ctx as never, () => ({ bindHost: '127.0.0.1', allowedHosts: [], tunnelHost: null })); + streamCoordinator = { + suspendSseTerminal: vi.fn(), + resumeSseTerminal: vi.fn(), + }; + registerWsRoutes( + app, + ctx as never, + () => ({ bindHost: '127.0.0.1', allowedHosts: [], tunnelHost: null }), + streamCoordinator + ); await app.listen({ port: PORT, host: '127.0.0.1' }); }); @@ -107,6 +135,43 @@ describe('ws-routes', () => { // ========== Terminal output ========== describe('terminal output', () => { + it('flushes queued output before handing the session stream back to SSE', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal?cid=page-client-a'); + try { + expect(streamCoordinator.suspendSseTerminal).toHaveBeenCalledWith('page-client-a', 'ws-test-session'); + + const messages = collectMessages(ws, 2); + ctx._session.emit('terminal', 'last websocket batch'); + ws.send('{"t":"h"}'); + + await expect(messages).resolves.toEqual([ + expect.objectContaining({ t: 'o', d: expect.stringContaining('last websocket batch') }), + { t: 'ha' }, + ]); + expect(streamCoordinator.resumeSseTerminal).toHaveBeenCalledWith('page-client-a', 'ws-test-session', false); + expect(ctx._session.listenerCount('terminal')).toBe(0); + } finally { + ws.close(); + } + }); + + it('requests recovery when handoff discards an incomplete ANSI suffix', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal?cid=page-client-a'); + try { + const messages = collectMessages(ws, 2); + ctx._session.emit('terminal', 'safe text\x1b['); + ws.send('{"t":"h"}'); + + await expect(messages).resolves.toEqual([ + expect.objectContaining({ t: 'o', d: expect.stringContaining('safe text') }), + { t: 'ha' }, + ]); + expect(streamCoordinator.resumeSseTerminal).toHaveBeenCalledWith('page-client-a', 'ws-test-session', true); + } finally { + ws.close(); + } + }); + it('receives terminal output via WS with DEC 2026 sync markers', async () => { const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); try { @@ -127,6 +192,29 @@ describe('ws-routes', () => { } }); + it('carries the terminal stream cursor with WebSocket output', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const cursor = { + stream: 'ws-cursor-stream', + generation: 3, + start: 40, + end: 51, + }; + ctx._session.emit('terminal', 'hello world', cursor); + + const msg = (await nextMessage(ws)) as { + t: string; + d: string; + cursor?: typeof cursor; + }; + expect(msg.t).toBe('o'); + expect(msg.cursor).toEqual(cursor); + } finally { + ws.close(); + } + }); + it('sends clearTerminal event as {"t":"c"}', async () => { const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); try { @@ -170,19 +258,215 @@ describe('ws-routes', () => { } }); - it('flushes immediately when batch exceeds size threshold', async () => { + it('coalesces adjacent mobile redraws across the desktop batch window', async () => { const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); try { const session = ctx._session; + ws.send(JSON.stringify({ t: 'z', c: 48, r: 28, v: 'mobile' })); + await vi.waitFor(() => { + expect(session.resize).toHaveBeenCalledWith(48, 28, { + viewportType: 'mobile', + force: false, + }); + }); - // Emit data larger than WS_BATCH_FLUSH_THRESHOLD (16384) - const largeData = 'X'.repeat(17000); - session.emit('terminal', largeData); + session.emit('terminal', 'mobile-frame-1'); + await new Promise((resolve) => setTimeout(resolve, 15)); + session.emit('terminal', 'mobile-frame-2'); - // Should flush immediately (no 8ms wait) — use a tight timeout - const msg = (await nextMessage(ws, 500)) as { t: string; d: string }; + const msg = (await nextMessage(ws)) as { t: string; d: string }; expect(msg.t).toBe('o'); - expect(msg.d).toContain(largeData); + expect(msg.d).toContain('mobile-frame-1mobile-frame-2'); + } finally { + ws.close(); + } + }); + + it('keeps large adjacent mobile redraw chunks in one synchronized transaction', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + ws.send(JSON.stringify({ t: 'z', c: 48, r: 28, v: 'mobile' })); + await vi.waitFor(() => { + expect(session.resize).toHaveBeenCalledWith(48, 28, { + viewportType: 'mobile', + force: false, + }); + }); + + const first = 'A'.repeat(24 * 1024); + const second = 'B'.repeat(24 * 1024); + const messagesPromise = collectMessages(ws, 3, 500); + session.emit('terminal', first); + await new Promise((resolve) => setTimeout(resolve, 15)); + session.emit('terminal', second); + + const messages = (await messagesPromise) as Array<{ t: string; d: string }>; + expect(synchronizedPayload(messages)).toBe(first + second); + expect(messages[0].d.startsWith(DEC_2026_START)).toBe(true); + expect(messages[0].d.endsWith(DEC_2026_END)).toBe(false); + expect(messages[1].d.includes(DEC_2026_START)).toBe(false); + expect(messages[1].d.includes(DEC_2026_END)).toBe(false); + expect(messages[2].d.startsWith(DEC_2026_START)).toBe(false); + expect(messages[2].d.endsWith(DEC_2026_END)).toBe(true); + } finally { + ws.close(); + } + }); + + it('splits mobile bulk output into one lossless synchronized transaction', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + ws.send(JSON.stringify({ t: 'z', c: 48, r: 28, v: 'mobile' })); + await vi.waitFor(() => { + expect(session.resize).toHaveBeenCalledWith(48, 28, { + viewportType: 'mobile', + force: false, + }); + }); + + const largeData = 'X'.repeat(24 * 1024); + const cursor = { + stream: 'mobile-bulk-stream', + generation: 4, + start: 100, + end: 100 + largeData.length, + }; + const messagesPromise = collectMessages(ws, 2, 500); + session.emit('terminal', largeData, cursor); + + const messages = (await messagesPromise) as Array<{ + t: string; + d: string; + cursor: typeof cursor; + }>; + const payloads = messages.map((message, index) => transportPayload(message, index, messages.length)); + expect(payloads.every((payload) => Buffer.byteLength(payload, 'utf8') <= 16 * 1024)).toBe(true); + expect(synchronizedPayload(messages)).toBe(largeData); + expect(messages[0].cursor.start).toBe(cursor.start); + expect(messages[0].cursor.end).toBe(messages[1].cursor.start); + expect(messages[1].cursor.end).toBe(cursor.end); + expect( + messages.every((message, index) => message.cursor.end - message.cursor.start === payloads[index].length) + ).toBe(true); + expect(messages[0].d.endsWith(DEC_2026_END)).toBe(false); + expect(messages[1].d.startsWith(DEC_2026_START)).toBe(false); + } finally { + ws.close(); + } + }); + + it('splits bulk UTF-8 output into bounded lossless frames', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + + const largeData = 'X'.repeat(8000) + '😀'.repeat(1000); + const messagesPromise = collectMessages(ws, 2, 500); + session.emit('terminal', largeData); + + const messages = (await messagesPromise) as Array<{ t: string; d: string }>; + const payloads = messages.map((message, index) => transportPayload(message, index, messages.length)); + expect(messages.every((message) => message.t === 'o')).toBe(true); + expect(payloads.every((payload) => Buffer.byteLength(payload, 'utf8') <= 8 * 1024)).toBe(true); + expect(synchronizedPayload(messages)).toBe(largeData); + } finally { + ws.close(); + } + }); + + it('does not split inside ANSI control sequences or synchronized updates', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + const colorSequence = '\x1b[38;5;196m'; + const firstPayload = 'X'.repeat(8190) + colorSequence; + const atomicPayload = '\x1b[?2026h' + 'Y'.repeat(9000) + '\x1b[?2026l'; + const messagesPromise = collectMessages(ws, 2, 500); + + session.emit('terminal', firstPayload + atomicPayload); + + const messages = (await messagesPromise) as Array<{ t: string; d: string }>; + const payloads = messages.map((message, index) => transportPayload(message, index, messages.length)); + expect(payloads[0]).toBe(firstPayload); + expect(payloads[1]).toBe(atomicPayload); + expect(synchronizedPayload(messages)).toBe(firstPayload + atomicPayload); + } finally { + ws.close(); + } + }); + + it('carries an incomplete ANSI suffix into the next PTY event', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + const messagesPromise = collectMessages(ws, 2, 500); + + session.emit('terminal', 'plain text\x1b[38;'); + await new Promise((resolve) => setTimeout(resolve, 20)); + session.emit('terminal', '5;196mRED'); + + const messages = (await messagesPromise) as Array<{ t: string; d: string }>; + const payloads = messages.map((message) => message.d.slice('\x1b[?2026h'.length, -'\x1b[?2026l'.length)); + expect(payloads).toEqual(['plain text', '\x1b[38;5;196mRED']); + expect(payloads.join('')).toBe('plain text\x1b[38;5;196mRED'); + } finally { + ws.close(); + } + }); + + it('does not carry an incomplete ANSI suffix across a cursor generation boundary', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + const messagesPromise = collectMessages(ws, 3, 500); + + session.emit('terminal', 'old text\x1b[', { + stream: 'cursor-stream', + generation: 1, + start: 0, + end: 11, + }); + await new Promise((resolve) => setTimeout(resolve, 20)); + session.emit('terminal', 'new text', { + stream: 'cursor-stream', + generation: 2, + start: 0, + end: 8, + }); + + const messages = (await messagesPromise) as Array<{ + t: string; + d?: string; + cursor?: { generation: number }; + }>; + expect(messages.map((message) => message.t)).toEqual(['o', 'r', 'o']); + expect(messages[0].d).toContain('old text'); + expect(messages[2].d).toContain('new text'); + expect(messages[2].d).not.toContain('\x1b[new text'); + expect(messages[2].cursor?.generation).toBe(2); + } finally { + ws.close(); + } + }); + + it('does not carry an incomplete ANSI suffix across clearTerminal', async () => { + const ws = await connectWs('/ws/sessions/ws-test-session/terminal'); + try { + const session = ctx._session; + const messagesPromise = collectMessages(ws, 3, 500); + + session.emit('terminal', 'before clear\x1b['); + await new Promise((resolve) => setTimeout(resolve, 20)); + session.emit('clearTerminal'); + session.emit('terminal', 'after clear'); + + const messages = (await messagesPromise) as Array<{ t: string; d?: string }>; + expect(messages.map((message) => message.t)).toEqual(['o', 'c', 'o']); + expect(messages[0].d).toContain('before clear'); + expect(messages[2].d).toContain('after clear'); + expect(messages[2].d).not.toContain('\x1b[after clear'); } finally { ws.close(); } diff --git a/test/session-listener-wiring.test.ts b/test/session-listener-wiring.test.ts index 74ea312c..5b4d6096 100644 --- a/test/session-listener-wiring.test.ts +++ b/test/session-listener-wiring.test.ts @@ -3,6 +3,18 @@ import { Session } from '../src/session.js'; import { createSessionListeners } from '../src/web/session-listener-wiring.js'; describe('session listener wiring', () => { + it('forwards terminal cursor metadata to the transport batcher', () => { + const session = new Session({ id: 'wiring-terminal-cursor-test', workingDir: '/tmp', mode: 'codex' }); + const batchTerminalData = vi.fn(); + const deps = { batchTerminalData } as unknown as Parameters[1]; + const cursor = { stream: 'stream-a', generation: 2, start: 10, end: 15 }; + + const refs = createSessionListeners(session, deps); + refs.terminal('chunk', cursor); + + expect(batchTerminalData).toHaveBeenCalledWith('wiring-terminal-cursor-test', 'chunk', cursor); + }); + it('forwards the attachment request source through registerAttachment', async () => { const session = new Session({ id: 'wiring-attach-source-test', workingDir: '/tmp', mode: 'codex' }); const registerAttachment = vi.fn(async () => undefined); diff --git a/test/sse-empty-terminal-filter.test.ts b/test/sse-empty-terminal-filter.test.ts new file mode 100644 index 00000000..9d25b361 --- /dev/null +++ b/test/sse-empty-terminal-filter.test.ts @@ -0,0 +1,103 @@ +import type { FastifyReply } from 'fastify'; +import { describe, expect, it } from 'vitest'; +import type { CleanupManager } from '../src/utils/index.js'; +import { SseStreamManager } from '../src/web/sse-stream-manager.js'; + +describe('SseStreamManager empty terminal filter', () => { + it('treats an empty session list as receive-none and null as receive-all', () => { + const manager = new SseStreamManager({ getSessionStateWithRespawn: () => null }, {} as CleanupManager); + const reply = { + raw: { + write: () => true, + once: () => {}, + }, + } as unknown as FastifyReply; + + manager.addClient(reply, null, false, 'client-filter-test'); + + expect(manager.updateClientFilter('client-filter-test', [])).toBe(true); + expect((manager as unknown as { sseClients: Map | null> }).sseClients.get(reply)).toEqual( + new Set() + ); + + expect(manager.updateClientFilter('client-filter-test', null)).toBe(true); + expect( + (manager as unknown as { sseClients: Map | null> }).sseClients.get(reply) + ).toBeNull(); + }); + + it('suppresses a WebSocket-owned session and resumes it with targeted recovery', () => { + const manager = new SseStreamManager({ getSessionStateWithRespawn: () => null }, {} as CleanupManager); + const writes: string[] = []; + const reply = { + raw: { + write: (data: string) => { + writes.push(data); + return true; + }, + once: () => {}, + }, + } as unknown as FastifyReply; + const internals = manager as unknown as { + terminalBatches: Map; + flushSessionTerminalBatch: (sessionId: string) => void; + }; + + manager.addClient(reply, new Set(['session-a']), false, 'page-client-a'); + manager.suspendClientSession('page-client-a', 'session-a'); + internals.terminalBatches.set('session-a', ['hidden while websocket owns output']); + internals.flushSessionTerminalBatch('session-a'); + expect(writes).toEqual([]); + + manager.resumeClientSession('page-client-a', 'session-a', true); + expect(writes.join('')).toContain('session:needsRefresh'); + + writes.length = 0; + internals.terminalBatches.set('session-a', ['visible after websocket handoff']); + internals.flushSessionTerminalBatch('session-a'); + expect(writes.join('')).toContain('visible after websocket handoff'); + }); + + it('coalesces contiguous cursor ranges into the terminal SSE event', () => { + const manager = new SseStreamManager({ getSessionStateWithRespawn: () => null }, {} as CleanupManager); + const writes: string[] = []; + const reply = { + raw: { + write: (data: string) => { + writes.push(data); + return true; + }, + once: () => {}, + }, + } as unknown as FastifyReply; + const internals = manager as unknown as { + flushSessionTerminalBatch: (sessionId: string) => void; + }; + + manager.addClient(reply, new Set(['session-a']), false, 'cursor-client'); + manager.batchTerminalData('session-a', 'abc', { + stream: 'stream-a', + generation: 3, + start: 10, + end: 13, + }); + manager.batchTerminalData('session-a', 'def', { + stream: 'stream-a', + generation: 3, + start: 13, + end: 16, + }); + internals.flushSessionTerminalBatch('session-a'); + + const dataLine = writes + .join('') + .split('\n') + .find((line) => line.startsWith('data: ')); + expect(dataLine).toBeDefined(); + expect(JSON.parse(dataLine!.slice(6))).toMatchObject({ + id: 'session-a', + data: expect.stringContaining('abcdef'), + cursor: { stream: 'stream-a', generation: 3, start: 10, end: 16 }, + }); + }); +}); diff --git a/test/sse-subscription-filter.test.ts b/test/sse-subscription-filter.test.ts index 59511d92..8b855561 100644 --- a/test/sse-subscription-filter.test.ts +++ b/test/sse-subscription-filter.test.ts @@ -107,6 +107,20 @@ describe('SSE Subscription Filtering', () => { expect((initEvent?.data as any).sessions).toBeDefined(); }); + it('should treat an explicitly empty sessions filter as terminal receive-none', async () => { + const collecting = collectSSEEvents(baseUrl, '?sessions=', 400); + await new Promise((resolve) => setTimeout(resolve, 100)); + ( + server as unknown as { + sse: { batchTerminalData: (sessionId: string, data: string) => void }; + } + ).sse.batchTerminalData('empty-filter-session', 'must-not-arrive'); + + const events = await collecting; + expect(events.find((event) => event.event === 'init')).toBeDefined(); + expect(events.find((event) => event.event === 'session:terminal')).toBeUndefined(); + }); + it('should receive all events when no sessions param is provided (backwards-compatible)', async () => { // Start two SSE listeners: one with no filter, one with a filter for a nonexistent session const controller1 = new AbortController(); diff --git a/test/tmux-history-page.test.ts b/test/tmux-history-page.test.ts new file mode 100644 index 00000000..5b7fbde8 --- /dev/null +++ b/test/tmux-history-page.test.ts @@ -0,0 +1,78 @@ +import { describe, expect, it, vi } from 'vitest'; +import { captureHistoryPageWithRunner, resolveHistoryPageRange } from '../src/tmux-manager.js'; + +describe('tmux history page ranges', () => { + it('returns the newest bounded page by default', () => { + expect(resolveHistoryPageRange(10_000, { limit: 1_000 })).toEqual({ + start: 9_000, + end: 10_000, + total: 10_000, + hasMoreBefore: true, + hasMoreAfter: false, + }); + }); + + it('walks backward and forward using absolute history rows', () => { + expect(resolveHistoryPageRange(10_000, { before: 9_000, limit: 1_000 })).toEqual({ + start: 8_000, + end: 9_000, + total: 10_000, + hasMoreBefore: true, + hasMoreAfter: true, + }); + expect(resolveHistoryPageRange(10_000, { after: 2_000, limit: 1_000 })).toEqual({ + start: 2_000, + end: 3_000, + total: 10_000, + hasMoreBefore: true, + hasMoreAfter: true, + }); + }); + + it('clamps empty and out-of-range requests without producing invalid tmux coordinates', () => { + expect(resolveHistoryPageRange(0, { limit: 1_000 })).toEqual({ + start: 0, + end: 0, + total: 0, + hasMoreBefore: false, + hasMoreAfter: false, + }); + expect(resolveHistoryPageRange(100, { before: 500, limit: 1_000 })).toEqual({ + start: 0, + end: 100, + total: 100, + hasMoreBefore: false, + hasMoreAfter: false, + }); + }); +}); + +describe('tmux history page capture', () => { + it('captures explicit physical-row ranges without joining wrapped rows', async () => { + const run = vi + .fn() + .mockResolvedValueOnce('10000\n') + .mockResolvedValueOnce('row-a\nrow-b\n') + .mockResolvedValueOnce('oldest-a\noldest-b\n'); + + const page = await captureHistoryPageWithRunner('%7', { before: 9000, limit: 1000 }, run); + + expect(page).toMatchObject({ + buffer: 'row-a\r\nrow-b', + start: 8000, + end: 9000, + total: 10000, + hasMoreBefore: true, + hasMoreAfter: true, + }); + expect(run.mock.calls[1][0]).toEqual(['capture-pane', '-p', '-e', '-S', '-2000', '-E', '-1001', '-t', '%7']); + expect(run.mock.calls[1][0]).not.toContain('-J'); + }); + + it('rejects invalid history-size output without issuing a capture', async () => { + const run = vi.fn().mockResolvedValue('not-a-number'); + + await expect(captureHistoryPageWithRunner('%7', { limit: 1000 }, run)).resolves.toBeNull(); + expect(run).toHaveBeenCalledTimes(1); + }); +}); diff --git a/test/ws-connection-registry.test.ts b/test/ws-connection-registry.test.ts index db1a9322..7c109d62 100644 --- a/test/ws-connection-registry.test.ts +++ b/test/ws-connection-registry.test.ts @@ -53,7 +53,7 @@ describe('WsConnectionRegistry', () => { expect(reg.register('s1', 'f', sock('f')).admitted).toBe(false); // Eagerly unregister one (simulating terminate/error, not async close). - reg.unregister('s1', sockets[0][1]); + expect(reg.unregister('s1', sockets[0][1])).toBe(true); expect(reg.liveCount('s1')).toBe(4); // Now a brand-new distinct client is admitted. @@ -92,7 +92,7 @@ describe('WsConnectionRegistry', () => { reg.register('s1', 'alice', fresh); // supersede // The stale socket's async close arrives late — must NOT remove fresh. - reg.unregister('s1', old); + expect(reg.unregister('s1', old)).toBe(false); expect(reg.liveCount('s1')).toBe(1); // Fresh is still the live entry: another reconnect evicts fresh, not old.