From d19d2b3e5797185a9312186f31ca95409328ab32 Mon Sep 17 00:00:00 2001 From: Vyncint Ng <115854244+vyncint@users.noreply.github.com> Date: Tue, 8 Sep 2026 22:07:01 +0700 Subject: [PATCH] test: termlens 0.10.1, and the invariant the PTY suite rests on MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The harness moves from 0.9. The upgrade itself is small — `drag` takes four column-first arguments instead of two coordinate pairs, so a transposed `find` result cannot be handed to it by mistake — but 0.10 brings the accessor this suite was missing. `crates/oxidelake-tui/tests/emulation.rs` pins `Screen::unsupported()` exactly. Every screen assertion in this repository reads a grid a VT emulator produced from the binary's bytes, so a sequence that emulator does not implement makes the grid quietly wrong and every assertion a claim about a plausible-looking fiction. The measured answer is one `SGR 59` — underline colour, which changes no cell. The four blink/strikethrough entries that termlens#320 reports as unsupported despite being implemented are noted in the test rather than pinned as if they were real. `crates/oxidelake-runtime/tests/oxide_tui_pty.rs` is the coverage gap this found. `oxidelake-tui` is tested through `oxidelake-tui-demo`, a binary that exists for the tests; what people install is `oxide`, and `oxide tui` renders the same model behind clap, tokio and a tracing subscriber writing to stderr. `tests/cli.rs` drives the other subcommands with assert_cmd, which captures pipes and cannot see that in a terminal stdout and stderr are one stream — so a log line lands on the dashboard and nothing was watching. `crates/oxidelake-tui/tests/termlens_cli.rs` drives termlens-cli at the version the lockfile names, so the tool and the library are one release. The vendored agent skill is refreshed to 0.10.1 and check-skill-version.sh fails when that copy and the dependency disagree on major.minor; it had drifted two releases with nothing to notice. `skill-version` is registered in ci-policy.py's always-run set, with tests. CI writes TERMLENS_ARTIFACT_DIR and renders failures into the job summary. Also gitignores __pycache__, which the Python scripts under .github/scripts have always produced and nothing excluded. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com> --- .claude/skills/termlens/SKILL.md | 138 +++++++-- .github/scripts/check-skill-version.sh | 53 ++++ .github/scripts/ci-policy.py | 2 +- .github/scripts/test_ci.py | 44 +++ .github/workflows/ci.yml | 50 ++- .github/workflows/stress.yml | 9 + .gitignore | 4 + AGENTS.md | 12 + CHANGELOG.md | 50 +++ Cargo.lock | 10 +- Cargo.toml | 7 +- Makefile | 22 +- STATUS.md | 2 +- crates/oxidelake-runtime/Cargo.toml | 11 + .../oxidelake-runtime/tests/oxide_tui_pty.rs | 149 +++++++++ crates/oxidelake-tui/Cargo.toml | 1 + crates/oxidelake-tui/tests/emulation.rs | 187 +++++++++++ ...e_first_frame_carries_its_colours_too.snap | 46 +++ crates/oxidelake-tui/tests/termlens_cli.rs | 292 ++++++++++++++++++ crates/oxidelake-tui/tests/tui_pty_test.rs | 222 ++++++++++++- docs/SPEC.md | 9 +- docs/decisions/ADR-0011-tui-testing-layers.md | 11 + docs/dependencies.md | 5 +- docs/verification.md | 30 +- 24 files changed, 1324 insertions(+), 42 deletions(-) create mode 100755 .github/scripts/check-skill-version.sh create mode 100644 crates/oxidelake-runtime/tests/oxide_tui_pty.rs create mode 100644 crates/oxidelake-tui/tests/emulation.rs create mode 100644 crates/oxidelake-tui/tests/snapshots/tui_pty_test__the_first_frame_carries_its_colours_too.snap create mode 100644 crates/oxidelake-tui/tests/termlens_cli.rs diff --git a/.claude/skills/termlens/SKILL.md b/.claude/skills/termlens/SKILL.md index 516115c..49394fa 100644 --- a/.claude/skills/termlens/SKILL.md +++ b/.claude/skills/termlens/SKILL.md @@ -5,7 +5,7 @@ description: Write, fix or review headless terminal tests for a Rust CLI or TUI # Testing terminal programs with termlens -Written against **termlens 0.9.0**. Every `rust` block below is a complete +Written against **termlens 0.10.1**. Every `rust` block below is a complete integration test that is compiled against the crate in CI, so the API it shows is the API that exists. The recipes spawn a binary called `myapp` that draws a list with a `> ` highlight, a status line ending in @@ -17,7 +17,10 @@ application's own texts where the comments say so. termlens spawns your **real binary** in a **real pseudo-terminal**, drains its output on a reader thread through a VT emulator into an in-memory **screen grid**, and lets a test wait on and assert against that grid — -Playwright for the terminal. Unix only (Linux, macOS). +Playwright for the terminal. Linux and macOS in full; on Windows (ConPTY) +screen assertions work and frame assertions do not — `wait_frame`, +`record`, graphics and mouse modes are Unix-only there, and a test that +needs one is `#[cfg_attr(windows, ignore = "…")]` with the reason. Use it for the things an in-process mock cannot see: @@ -109,9 +112,11 @@ your test ── send(Key) · click · paste · resize ──▶ PTY └─ 8. **`wait_frame` only works for applications that emit DEC 2026 synchronized updates.** Stock ratatui 0.30 with crossterm does **not** (measured: `repaints()` stays 0), so `wait_frame` times out against it - with a message saying exactly that. Default to `snapshot_after`. Use - `wait_frame` only if the application brackets its repaints in - `BeginSynchronizedUpdate` / `EndSynchronizedUpdate`. + with a message saying exactly that, and `Terminal::record()` refuses + for the same reason. Default to `snapshot_after`. Use `wait_frame` only + if the application brackets its repaints in `BeginSynchronizedUpdate` / + `EndSynchronizedUpdate` — then a `wait_frame` timeout also shows the + diff from the last frame it returned to the live screen. 9. **Return `termlens::Result<()>` from the test and use `?`.** The `Display` of every error carries the screen, so a failing wait prints @@ -120,9 +125,11 @@ your test ── send(Key) · click · paste · resize ──▶ PTY └─ 10. **Snapshot the `Screen`, not its text.** `insta::assert_snapshot!(screen)` records the header (`size: 80x24 cursor: 3,5` or `cursor: hidden`) and the grid; `screen.with_styles()` adds a `styles:` block that catches a - colour regression. `.text()` drops the header and `format!("{:?}")` is - the same as `Display`. Review changes with `cargo insta review`; never - blind-accept with `INSTA_UPDATE=always`. + colour regression. The one-liner that gets all three decisions right — + wait, settle, styles — is `termlens::assert_screen_snapshot!(t, after = + |s| s.contains("Ready"))`. `.text()` drops the header and + `format!("{:?}")` is the same as `Display`. Review changes with `cargo + insta review`; never blind-accept with `INSTA_UPDATE=always`. 11. **The environment is hermetic by default — set what the app reads.** Under `env_clear()` (which `bin!` applies) the child sees only @@ -136,26 +143,36 @@ your test ── send(Key) · click · paste · resize ──▶ PTY └─ (CJK, most emoji) occupies two cells: the leading one `is_wide()`, the next `is_wide_continuation()`. `find` reports real terminal columns. `contains` and `find` fold both sides to NFC and search the **visible - screen only** — text that scrolled off is in `full_text()`, and a line - that wrapped is two rows, so a needle spanning the wrap is not found. + screen only** — text that scrolled off is in `full_text()` (and + `locate` says which region holds a needle), and a line that wrapped is + two rows, so a needle spanning the wrap is found by `logical_text()`, + not by `contains`. `find_all` lists every match; a volatile cell is + masked in the grid with `mask_rect` / `mask_matching`, never edited in + the text. ## 4. Setup ```toml [dev-dependencies] -termlens = "0.9" +termlens = "0.10" insta = "1" # for the snapshot recipes; termlens also re-exports it as `termlens::insta` ``` +Features, all off by default except `insta`: `regex` (a pattern over a row +of the screen: `wait_until_matches`, `find_match`, `mask_matches`), +`serde` (a `Screen` as JSON and back, for `assert_json_snapshot!` or a CI +step), `decode` (the pixels of inline images). + - Put the tests in `tests/` **of the package that owns the `[[bin]]`**: Cargo sets `CARGO_BIN_EXE_` only there, and `bin!` needs it at compile time. For a binary in a sibling crate, build it and pass the path to `Terminal::builder().spawn(path)` instead. - The binary is built by `cargo test` before the tests run. Tests run in parallel by default; each spawns its own PTY, which is fine. -- Gate the test file with `#![cfg(unix)]` if the crate must also build on - Windows. -- `add --features decode` only if you assert on the pixels of inline images. +- The crate builds and runs on Windows over ConPTY. Do not gate whole files + with `#![cfg(unix)]`; mark the individual tests the platform cannot + honour — frames, graphics, mouse modes, signals — with + `#[cfg_attr(windows, ignore = "…")]` naming the reason. ## 5. Recipes @@ -318,6 +335,54 @@ fn cells_styles_and_wide_characters() -> termlens::Result<()> { } ``` +### Recipe E — the snapshot macro, a cell diff, and a masked clock + +```rust +use termlens::{Key, Screen}; + +#[test] +fn snapshot_diff_and_mask() -> termlens::Result<()> { + let mut t = termlens::bin!("myapp")?; + + // Wait for the marker, let the picture settle, snapshot WITH styles: + // the three decisions every TUI snapshot needs, in one line. `styles = + // false` for text only; `(&screen)` snapshots a Screen you already hold. + termlens::assert_screen_snapshot!(t, after = |s| s.contains("Ready")); + + // Two screens, and what changed between them — rows, columns, style + // runs — rendered in the assertion message rather than two whole grids. + let before = t.screen(); + t.send(Key::Char('j'))?; + let after = t.snapshot_after(|s| s.contains("> Beta"))?; + let diff = before.diff(&after); + assert!(!diff.is_empty(), "j should have moved the highlight:\n{diff}"); + assert!(diff.cells().any(|(row, _, _, _)| row == 1), "{diff}"); + + // A clock in the top-right corner breaks whole-screen snapshots. Mask + // it in the GRID — the mask keeps every column and style where it was, + // which a text filter over the rendering cannot. (cols, rows), like size(). + let masked: Screen = after.mask_rect(70..80, 0..1); + // `mask_matching` takes a LITERAL, not a set of characters: this hides + // the exact text "Counter: 1". To mask by shape, use `mask_cells` (or + // `mask_matches` with the `regex` feature) — a predicate per cell. + let _by_text = after.mask_matching("Counter: 1", '#'); + let _digits_hidden = after.mask_cells(|c| { + !c.contents().is_empty() && c.contents().chars().all(|ch| ch.is_ascii_digit()) + }); + termlens::assert_screen_snapshot!(&masked); + + // Every occurrence, not just the first; and a needle that spans a soft + // wrap, which contains() cannot see across rows. + let separators = after.find_all("│"); + assert!(separators.len() >= 2, "{after}"); + assert!(after.logical_text().contains("Ready: j/k move, q quits")); + + t.send(Key::Char('q'))?; + assert!(t.wait_exit()?.success()); + Ok(()) +} +``` + ## 6. Reading a failure Every error's `Display` ends with the screen, under a header that says @@ -330,7 +395,8 @@ Read the first line for the cause: | `… note: N rows have scrolled off the top` | the text went into history | assert with `full_text()` / `scrollback_text()` | | `… note: the application queried the terminal (^[[?u …) and received no answer` | the app is blocked on a probe termlens deliberately does not answer | the app needs a fallback; see the termlens README's Known limitations | | `terminal closed (EOF) while waiting for …` | the app exited before the predicate held | check `wait_exit()` first, or the app crashed — the final screen shows why | -| `the application never emitted a DEC 2026 synchronized update` | `wait_frame` against an app without synchronized output | use `snapshot_after` / `wait_until` (rule 8) | +| `the application never emitted a DEC 2026 synchronized update` | `wait_frame` or `record().stop()` against an app without synchronized output | use `snapshot_after` / `wait_until` (rule 8) | +| `--- last returned frame → live screen ---` under a `wait_frame` timeout | the app repainted, but never into the predicate | the diff shows what did change; the predicate is looking at the wrong thing | | `input not receivable: the application has not enabled mouse tracking` | `click`/`drag`/`scroll` before the app enabled the mouse | `wait_until(|s| s.mouse_mode() != MouseMode::None)` first | | `input not receivable: mouse at (50, 2) is outside the 20x5 grid` | coordinates swapped or out of range | rule 6 | | `failed to spawn \`sh\`: \`sh\` is a bare program name and env_clear() removed PATH` | bare program name under `env_clear` | absolute path, `bin!`, or `.env("PATH", …)` | @@ -350,6 +416,8 @@ Read the first line for the cause: | `.env(k, v)` / `.envs([..])` / `.env_clear()` | environment; `env_clear` keeps `TERM` and `SHELL` pinned and drops the rest | | `.current_dir(path)` | default: the test process's directory | | `.scrollback(rows)` | history retained (default 1000, text only) | +| `.scrollback_styles(true)` | retain scrolled rows as cells too, so `scrollback_cell` keeps a masked-field assertion alive after it scrolls (measured cost in the rustdoc) | +| `.record_budget(cells)` | how much `record()` retains before dropping the oldest frames | | `.spawn(program) -> Result` | program is a path or a name on `PATH` | **Wait** (all return `termlens::Result`, all embed the screen on failure, all have a `_for(…, timeout)` twin): @@ -362,6 +430,8 @@ Read the first line for the cause: | `wait_idle(quiet)` | `()` | no *bytes* for `quiet` — a weaker, older sibling of `wait_stable` | | `wait_frame(\|s\| bool)` | `Screen` | complete DEC 2026 frames only (rule 8) | | `wait_exit()` | `ExitStatus` | the child's exit; `success()`, `code() -> Option`, `signal() -> Option<&str>` | +| `wait_until_matches(&Regex)` | `Screen` | feature `regex`: a pattern over a row of the screen — the expect-style wait, on the grid | +| `record()` … `.stop()` | `Recording` | every complete DEC 2026 frame with its time; `frames()`, `write_asciicast(path)` for a file `asciinema` plays | **Drive**: `send(Key)`, `send_str("text")` (no Enter — send `Key::Enter` yourself; `"\n"` would send LF, not CR), `paste("text")` (bracketed if the @@ -379,17 +449,24 @@ from_r, to_c, to_r)`, `scroll(col, row, Scroll::Down)`, `resize(cols, rows)`, | Accessor | Returns | |---|---| -| `contains(&str)` / `find(&str)` | `bool` / `Option<(row, col)>` — visible grid, NFC-folded | +| `contains(&str)` / `find(&str)` / `find_all(&str)` | `bool` / `Option<(row, col)>` / `Vec<(row, col)>` — visible grid, NFC-folded | +| `locate(&str)` | `Option`: `Screen { row, col }` or `History { row, col }` | +| `logical_text()` / `row_wrapped(row)` | wrapped rows joined back into lines / where the backend wrapped | | `find_by(\|&Cell\| bool)` | `Option<(row, col)>` | | `cell(row, col)` | `Option<&Cell>`: `contents()`, `style()`, `is_wide()`, `is_wide_continuation()` | | `row_text(row)` / `text()` / `rect_text(cols, rows)` | `String` | | `full_text()` / `scrollback_text()` / `scrollback_rows()` | history + screen / history / count | +| `scrollback_cell(row, col)` / `styled_scrollback()` | history as cells, with `scrollback_styles(true)` | | `size()` / `cols()` / `rows()` | `(cols, rows)` | | `cursor()` | `(row, col, visible)`; `cursor_shape()`, `cursor_blink()` | | `alternate_screen()`, `bracketed_paste()`, `application_cursor()`, `focus_events()` | mode flags | | `mouse_mode()` / `mouse_modes()` | reporting protocol / the set the app enabled | | `title()`, `clipboard()`, `links()`, `bells()`, `repaints()`, `graphics()` | out-of-band state | +| `unsupported()` / `insert_mode()` | sequences the emulator did not implement (`^[[20h`…), so a plausible grid can be told from a right one / IRM left on | | `with_styles()` | `Display` with a `styles:` block; snapshot this to catch colour regressions | +| `diff(&other)` | `ScreenDiff`: `is_empty()`, `cells()`, and a `Display` of only the rows that changed | +| `mask_rect(cols, rows)` / `mask_matching(literal, fill)` / `mask_cells(pred)` | a new `Screen` with those cells replaced, styles and columns intact. `mask_matching` is a literal (rows included — it spans a wrap the way `find_all` does); `mask_cells` blanks by predicate | +| `to_ansi()` / `to_svg()` / `to_html()` | renderings a person can see; `Screen::parse(text)` reads the text format back | **Style** (`Copy`, public fields): `fg`, `bg` (`Color::Default` / `Color::Indexed(u8)` / `Color::Rgb(u8, u8, u8)`), `bold`, `dim`, `italic`, @@ -399,8 +476,10 @@ double underline are not modelled. **Errors** (`termlens::Error`, `#[non_exhaustive]`): `Timeout { waiting_for, timeout, screen }`, `Eof { waiting_for, screen }`, `Spawn { command, reason }`, `Size(String)`, `Input(String)`, `Write { what, screen }`, `Emulator { -detail, screen }`, `Pty(String)`, `Io(std::io::Error)`. `err.screen()` returns -the embedded screen when there is one. +detail, screen }`, `Parse(String)`, `Pty(String)`, `Io(std::io::Error)`. +`err.screen()` returns the embedded screen when there is one. With +`TERMLENS_ARTIFACT_DIR` set, every such screen is also written to that +directory (see §9b). ## 8. Pitfalls an agent falls into, and the fix @@ -431,9 +510,13 @@ the embedded screen when there is one. - `insta::assert_snapshot!("name", screen)` — several snapshots in one test. - Inline snapshots work: `insta::assert_snapshot!(screen, @"")`, then `cargo insta review` fills the literal. -- `termlens::assert_screen_snapshot!(screen)` is the same call through the - `insta` termlens re-exports, for crates that do not want their own `insta` - dev-dependency. +- `termlens::assert_screen_snapshot!(t)` settles the terminal (100 ms of + stillness) and snapshots **with styles**; `(t, after = |s| …)` waits for + the predicate first; `(t, styles = false)` for text only; `(&screen)` + for a `Screen` already in hand; `(t, @"…")` inline. Through the `insta` + termlens re-exports, so no separate `insta` dev-dependency is needed. +- With the `serde` feature, `insta::assert_json_snapshot!(screen)` records + the structured form and takes insta's redactions per field. - Volatile content (a clock, a PID, a spinner) breaks whole-screen snapshots. insta's text filters are not grid-aware — a shorter replacement shifts every column after it — so prefer asserting the stable region with @@ -442,6 +525,19 @@ the embedded screen when there is one. - Snapshot files live in `tests/snapshots/`; commit them. Review every change with `cargo insta review`; a diff you cannot explain is a bug. +## 9b. At a shell prompt, and in CI + +- `cargo install termlens-cli` gives the harness as a command: `termlens + inspect --size 120x40 myapp` prints what a program shows (`--ansi` for + colour), `termlens diff old.snap new.snap.new` prints the cell diff of two + saved screens and exits 1 if anything changed, `termlens render --svg + failing.snap` makes an image. A saved screen is any text termlens prints + — an insta `.snap`, the grid a wait error leaves in a log. +- In CI, set `TERMLENS_ARTIFACT_DIR: ${{ runner.temp }}/termlens` on the + test step and add `uses: vyncint/termlens/.github/actions/report@v0.10.0` + with `if: failure()` after it: every screen a failing wait embedded, and + every `.snap.new` with its diff, lands in the pull request's step summary. + ## 10. A checklist before you finish - [ ] No `sleep` anywhere; every wait names what it waits for. diff --git a/.github/scripts/check-skill-version.sh b/.github/scripts/check-skill-version.sh new file mode 100755 index 0000000..bdc2e19 --- /dev/null +++ b/.github/scripts/check-skill-version.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# The vendored termlens skill must name the version this workspace depends on. +# +# `.claude/skills/termlens/SKILL.md` is a copy of the file termlens ships for +# coding agents. It is refreshed by hand, and the failure mode is silent: the +# dependency gets bumped, the copy does not, and every agent working in this +# repository is then handed guidance for a version that is no longer here — +# wrong signatures, absent APIs, advice that was true one release ago. That +# is not hypothetical here: the copy sat at 0.9.0 while `docs/` still said +# 0.6, three releases behind the dependency. +# +# Nothing can diff it against upstream: the published crate does not ship the +# skill, so there is no registry copy to compare with. What *is* checkable is +# that the two versions agree, which is exactly the drift that happens. +# +# Compares major.minor only. A termlens patch release does not rewrite the +# skill, and demanding a re-copy for every one of them would make this noise. +# +# Usage: check-skill-version.sh [SKILL.md] [Cargo.toml] +set -euo pipefail +cd "$(dirname "$0")/../.." + +skill="${1:-.claude/skills/termlens/SKILL.md}" +manifest="${2:-Cargo.toml}" + +[ -f "$skill" ] || { echo "::error::$skill does not exist"; exit 1; } +[ -f "$manifest" ] || { echo "::error::$manifest does not exist"; exit 1; } + +# "Written against **termlens 0.10.1**." -> 0.10.1 +skill_version="$(sed -n 's/.*Written against \*\*termlens \([0-9][0-9.]*\)\*\*.*/\1/p' "$skill" | head -1)" +[ -n "$skill_version" ] || { + echo "::error::$skill has no 'Written against **termlens X.Y.Z**' line to check" + exit 1 +} +skill_minor="$(echo "$skill_version" | cut -d. -f1,2)" + +# In [workspace.dependencies], either spelling: +# termlens = "0.10" +# termlens = { version = "0.10", features = [...] } +dep_version="$(sed -n 's/^termlens = .*version = "\([0-9][0-9.]*\)".*/\1/p;s/^termlens = "\([0-9][0-9.]*\)".*/\1/p' "$manifest" | head -1)" +[ -n "$dep_version" ] || { + echo "::error::no termlens dependency with a version found in $manifest" + exit 1 +} +dep_minor="$(echo "$dep_version" | cut -d. -f1,2)" + +if [ "$skill_minor" != "$dep_minor" ]; then + echo "::error::the vendored termlens skill is written against ${skill_version} but this workspace depends on ${dep_version}." + echo "::error::Refresh it: cp ../termlens/skills/termlens/SKILL.md ${skill}" + exit 1 +fi + +echo "the vendored termlens skill (${skill_version}) matches the dependency (${dep_version})" diff --git a/.github/scripts/ci-policy.py b/.github/scripts/ci-policy.py index 985c696..8bba1c4 100644 --- a/.github/scripts/ci-policy.py +++ b/.github/scripts/ci-policy.py @@ -10,7 +10,7 @@ RUST_JOBS = {"fmt", "clippy", "test", "metal", "msrv", "docs", "deny"} -ALWAYS_JOBS = {"changes", "release-scripts", "zizmor"} +ALWAYS_JOBS = {"changes", "release-scripts", "skill-version", "zizmor"} ROOT_DOCS = {"CHANGELOG.md", "CONTRIBUTING.md", "AGENTS.md", "CODE_OF_CONDUCT.md", "SECURITY.md"} diff --git a/.github/scripts/test_ci.py b/.github/scripts/test_ci.py index 72efb96..17e3b1a 100644 --- a/.github/scripts/test_ci.py +++ b/.github/scripts/test_ci.py @@ -221,6 +221,50 @@ def test_empty_fields_wrong_docs_and_missing_readme_fail(self): self.assertNotEqual(self.run_metadata(packaged=False).returncode, 0) +class SkillVersionTests(unittest.TestCase): + """The vendored termlens skill must name the dependency's major.minor.""" + + def run_check(self, skill_version, dep_line): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + scripts = root / ".github/scripts" + scripts.mkdir(parents=True) + script = scripts / "check-skill-version.sh" + script.write_text((SCRIPTS / script.name).read_text()) + skill = root / ".claude/skills/termlens/SKILL.md" + skill.parent.mkdir(parents=True) + header = f"Written against **termlens {skill_version}**.\n" if skill_version else "" + skill.write_text("# Testing terminal programs with termlens\n\n" + header) + (root / "Cargo.toml").write_text(f"[workspace.dependencies]\n{dep_line}\n") + return subprocess.run(["/bin/bash", str(script)], text=True, capture_output=True, cwd=directory) + + def test_a_matching_skill_passes_and_a_patch_release_is_not_drift(self): + for skill, dep in (("0.10.1", 'termlens = { version = "0.10", features = ["serde"] }'), + ("0.10.0", 'termlens = "0.10"'), + ("0.10.7", 'termlens = { version = "0.10.1" }')): + with self.subTest(skill=skill, dep=dep): + run = self.run_check(skill, dep) + self.assertEqual(run.returncode, 0, run.stdout + run.stderr) + + def test_drift_fails_in_both_directions_and_says_so(self): + for skill, dep in (("0.9.0", 'termlens = { version = "0.10", features = ["serde"] }'), + ("0.11.0", 'termlens = "0.10"'), + ("1.0.0", 'termlens = "0.10"')): + with self.subTest(skill=skill, dep=dep): + run = self.run_check(skill, dep) + self.assertNotEqual(run.returncode, 0) + self.assertIn("::error::", run.stdout) + + def test_an_unreadable_claim_is_a_failure_not_a_pass(self): + self.assertNotEqual(self.run_check("", 'termlens = "0.10"').returncode, 0) + self.assertNotEqual(self.run_check("0.10.1", "insta = \"1\"").returncode, 0) + + def test_the_committed_skill_matches_the_committed_manifest(self): + run = subprocess.run(["/bin/bash", str(SCRIPTS / "check-skill-version.sh")], + text=True, capture_output=True, cwd=ROOT) + self.assertEqual(run.returncode, 0, run.stdout + run.stderr) + + class ToolchainTests(unittest.TestCase): def test_msrv_uses_rustup_even_when_path_cargo_is_not_a_proxy(self): with tempfile.TemporaryDirectory() as directory: diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a0086b6..ea5db60 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -143,7 +143,23 @@ jobs: - name: Test the selected feature configuration env: TARGET: ${{ matrix.lane.target }} + # Every screen a failing termlens wait carries is also written + # here, so the `report` step below can put it in the job summary + # instead of leaving it in a log someone has to scroll. The TUI + # PTY suite is a flake target (that is what stress.yml is for) and + # a flake in CI used to leave nothing but scrollback (termlens + # 0.10). + TERMLENS_ARTIFACT_DIR: ${{ runner.temp }}/termlens run: make "$TARGET" + # `#[ignore]`d so a plain `cargo test` never installs anything behind a + # contributor's back — these crates are published. CI is where they + # should run, so CI asks for them by name. The install is the version + # Cargo.lock names, so the tool and the library are one release. + - name: termlens-cli against the committed screens + if: matrix.lane.name == 'default' + env: + TERMLENS_ARTIFACT_DIR: ${{ runner.temp }}/termlens + run: make test-termlens-cli - name: predict adds no second CUDA stack if: matrix.lane.name == 'predict' run: make check-predict-no-second-cuda @@ -153,6 +169,14 @@ jobs: - name: dependency chain is coherent if: matrix.lane.name == 'default' run: make coherence + # A PTY failure is a picture, and a log is the worst place to read one. + # This renders every screen the suite left behind — and every insta + # `.snap.new` — into the job summary, with SVG and HTML uploaded. It + # installs termlens-cli itself, so nothing else here has to. + - uses: vyncint/termlens/.github/actions/report@e1b96c8203fd727fa3458af395719c429966ee82 # v0.10.1 + if: failure() + with: + name: termlens-report-linux-${{ matrix.lane.name }} - name: Upload Cargo build timings if: always() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 @@ -186,9 +210,19 @@ jobs: cache-workspace-crates: false - run: make lint-metal - name: Metal package tests and macOS PTY suite + env: + TERMLENS_ARTIFACT_DIR: ${{ runner.temp }}/termlens run: make test-metal - name: conformance suite on the Metal device + env: + TERMLENS_ARTIFACT_DIR: ${{ runner.temp }}/termlens run: make test-metal-device + # macOS is the leg where the PTY suite times out first, and the one + # nobody can reproduce locally on a Linux box. + - uses: vyncint/termlens/.github/actions/report@e1b96c8203fd727fa3458af395719c429966ee82 # v0.10.1 + if: failure() + with: + name: termlens-report-metal - name: Upload Cargo build timings if: always() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 @@ -268,6 +302,20 @@ jobs: toolchain: stable - run: ./.github/scripts/check-crate-metadata.sh + # The vendored termlens skill is refreshed by hand, and a stale copy is + # silent: agents working here get guidance for a version that is no longer + # the dependency. Nothing can diff it against upstream — the published + # crate does not ship the skill — but the two versions agreeing is + # checkable, and that is the drift that happens (`make skill-version`). + skill-version: + name: skill version + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + with: + persist-credentials: false + - run: make skill-version + # Workflow security audit: unpinned actions, credential persistence, # template injection. Accepted findings live in .github/zizmor.yml. zizmor: @@ -287,7 +335,7 @@ jobs: required-green: name: required-green if: always() - needs: [changes, fmt, clippy, test, metal, msrv, docs, deny, release-scripts, zizmor] + needs: [changes, fmt, clippy, test, metal, msrv, docs, deny, release-scripts, skill-version, zizmor] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 diff --git a/.github/workflows/stress.yml b/.github/workflows/stress.yml index c6908ed..1b3e974 100644 --- a/.github/workflows/stress.yml +++ b/.github/workflows/stress.yml @@ -43,7 +43,16 @@ jobs: - name: Build once and exercise every thread count env: ITERS: ${{ inputs.iterations || '100' }} + # The whole point of this workflow is the one iteration in a hundred + # that fails, and a screen in a 45-minute log is the hardest place + # to read one. Every screen a failing wait carries is written here + # for the report step below (termlens 0.10). + TERMLENS_ARTIFACT_DIR: ${{ runner.temp }}/termlens run: make stress-tui + - uses: vyncint/termlens/.github/actions/report@e1b96c8203fd727fa3458af395719c429966ee82 # v0.10.1 + if: failure() + with: + name: termlens-report-stress-${{ matrix.os }} - name: Upload Cargo build timings if: always() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 diff --git a/.gitignore b/.gitignore index 0a141e7..7fad5c0 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,10 @@ /data/ /spill/ +# Python bytecode from the scripts under .github/scripts +__pycache__/ +*.pyc + # editors / OS .DS_Store .idea/ diff --git a/AGENTS.md b/AGENTS.md index ad76d0e..99972e6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,18 @@ lockfile. - **The TUI emits DEC 2026 synchronized updates**, which most ratatui apps do not — so its PTY tests use termlens's `wait_frame`, not `snapshot_after`. A snapshot taken any other way can catch a half-painted screen, and did. +- **`.claude/skills/termlens/SKILL.md` is a vendored copy, and `make gate` + checks its version against the dependency.** Bumping `termlens` in + `Cargo.toml` means copying the skill over in the same change + (`cp ../termlens/skills/termlens/SKILL.md .claude/skills/termlens/SKILL.md`); + the `skill-version` job fails otherwise. Guidance for a version that is no + longer here is worse than none. +- **The three text screen snapshots are text on purpose.** termlens 0.10 + records styles by default; `tui_pty_test.rs` passes `styles = false` there + and keeps the colours in one styled snapshot plus direct cell assertions, + so a colour change does not rewrite three large files. `emulation.rs` pins + what the emulator could not render — read it before trusting a screen + assertion that suddenly went green. ## The rules that will fail CI diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a7a0fb..3f3117d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,56 @@ versions (0.x) may contain breaking changes; they are always listed under a ## [Unreleased] +### Changed + +- **The PTY test harness moved to termlens 0.10.1** (from 0.9). The three + committed screen snapshots are unchanged: `assert_screen_snapshot!` records + styles by default in 0.10, and the text snapshots opt out with + `styles = false` so a colour change lands in one new styled snapshot rather + than rewriting three large files. The vendored agent skill + (`.claude/skills/termlens/SKILL.md`) was refreshed to match and is now + checked against the dependency by `make skill-version`, a CI job — it had + drifted two releases behind without anything noticing. + +### Added + +- **The dashboard's colours are tested.** Nothing asserted them before: + `TestBackend::to_string()` is text-only and the PTY snapshots were plain, + so the focused panel's yellow border and the green/blue backend tags were + invisible to the whole suite. `[CUDA]` is now green and `[CPU]` blue at + *every* occurrence (`Screen::find_all`), `Tab` is asserted to move the + highlight without changing one character of text, and `↓` is asserted to + leave the telemetry gauges and the Describe table untouched + (`Screen::diff`). + +- **`crates/oxidelake-tui/tests/emulation.rs`**: the invariant the rest of the + PTY suite rests on. `Screen::unsupported()` is pinned to exactly + `["^[[59m"]` — ratatui's underline-colour reset, which changes no cell — so + a sequence the emulator silently drops can no longer make every screen + assertion true against a wrong grid. Insert mode, bells, wrapped rows and + mouse modes are pinned beside it, and a dashboard screen is round-tripped + through the snapshot text format and through JSON. + +- **`crates/oxidelake-runtime/tests/oxide_tui_pty.rs`**: `oxide tui` — the + dashboard as users install it — in a real PTY. It renders the same frame + `oxidelake-tui` snapshots, it gives the terminal back, and the `tracing` + subscriber writing to stderr (the dashboard's own stream in a terminal) + puts nothing on the grid, `RUST_LOG=info` included. `assert_cmd` captures + pipes and could see none of that. + +- **`crates/oxidelake-tui/tests/termlens_cli.rs`**: the committed `.snap` + files read back with `termlens-cli` — `render --text/--svg/--html` keeping + the palette, and `diff`'s 0/1/2 exit codes on this repository's own + screens. `#[ignore]`d, because a published crate's `cargo test` must not + install a tool behind a contributor's back; CI runs it as + `make test-termlens-cli`. + +- **CI renders a failing PTY screen instead of logging it.** The Linux and + macOS test lanes run with `TERMLENS_ARTIFACT_DIR` set, and on failure the + pinned `vyncint/termlens` report action writes every screen the suite left + behind — and every insta `.snap.new` — into the job summary, with SVG and + HTML uploaded. + ### Fixed - **The dashboard's Describe panel no longer prints a truncated percentile as diff --git a/Cargo.lock b/Cargo.lock index 49b6b71..d9e3f95 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3316,6 +3316,7 @@ checksum = "86f0f8fee8c926415c58d6ae43a08523a26faccb2323f5e6b644fe7dd4ef6b82" dependencies = [ "console", "once_cell", + "serde", "similar", "tempfile", ] @@ -4146,6 +4147,7 @@ dependencies = [ "oxidelake-tui", "parquet", "tempfile", + "termlens", "tokio", "tracing", "tracing-subscriber", @@ -4178,6 +4180,7 @@ dependencies = [ "insta", "oxidelake-core", "ratatui", + "serde_json", "termlens", ] @@ -5771,15 +5774,18 @@ dependencies = [ [[package]] name = "termlens" -version = "0.9.0" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0fca989672430e13284b48504c44b499d88f5b39cb36b0e8dc5b45f06df50b09" +checksum = "c01f5cc1f410aa421987ca194b72620037422bddcfeb15b1f8da40e52b03b348" dependencies = [ "insta", "libc", "portable-pty", + "serde", + "serde_json", "thiserror 2.0.20", "unicode-normalization", + "unicode-width", "vt100", ] diff --git a/Cargo.toml b/Cargo.toml index 9e4dd85..d1cda55 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -83,10 +83,15 @@ safetensors = "0.8" # reading a model's header to recover its s # test-only insta = "1" -termlens = "0.9" +# `serde` is on because the PTY suite round-trips a screen through JSON — the +# exact bytes `TERMLENS_ARTIFACT_DIR` writes when a wait fails in CI, so the +# artefact a failure leaves behind is a format this suite has read back. It +# also makes those artefacts `.screen.json` rather than `.screen.txt`. +termlens = { version = "0.10", features = ["serde"] } assert_cmd = "2" tempfile = "3" rand = "0.10" +serde_json = "1" # reading a serialized termlens Screen back [workspace.lints.rust] unsafe_op_in_unsafe_fn = "deny" diff --git a/Makefile b/Makefile index 13098a1..ca16b07 100644 --- a/Makefile +++ b/Makefile @@ -15,7 +15,7 @@ UNAME := $(shell uname -s) # selecting compute alone rebuilt Arrow/DataFusion for the second pass in CI. METAL_TEST_ARGS := -p oxidelake-memory -p oxidelake-device -p oxidelake-compute -p oxidelake-runtime -p oxidelake-tui --features oxidelake-runtime/metal --locked --timings -.PHONY: help fmt fmt-check lint lint-cuda lint-metal lint-predict test test-predict test-io-uring test-metal test-metal-device check-cuda check-predict-no-second-cuda doc coherence deny gate metal msrv quickstart clean-data release-scripts crate-metadata zizmor ci-scripts stress-tui +.PHONY: help fmt fmt-check lint lint-cuda lint-metal lint-predict test test-predict test-io-uring test-termlens-cli test-metal test-metal-device check-cuda check-predict-no-second-cuda doc coherence deny gate metal msrv quickstart clean-data release-scripts crate-metadata skill-version zizmor ci-scripts stress-tui help: ## list targets @grep -E '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " %-14s %s\n", $$1, $$2}' @@ -73,12 +73,25 @@ test: ## the full default-feature test suite test-io-uring: ## io_uring object store tests (Linux; skips with a reason where io_uring is denied) $(CARGO) test -p oxidelake-storage --features io-uring --locked --timings +# Not part of `make gate`: it installs termlens-cli from crates.io at the +# version Cargo.lock names, and the gate has to run on a machine with no +# network. CI runs it in the default Linux test lane (docs/verification.md). +# The tests are #[ignore]d for the same reason — a `cargo test` on a +# published crate must not install anything behind a contributor's back. +test-termlens-cli: ## the termlens-cli suite against the committed screens (installs termlens-cli) + $(CARGO) test -p oxidelake-tui --test termlens_cli --locked -- --ignored + test-metal: ## Metal package tests and macOS PTY tests with one dependency graph $(CARGO) test $(METAL_TEST_ARGS) +# `--skip termlens_cli` keeps this pass off the network: the graph includes +# -p oxidelake-tui, whose `termlens_cli` tests are #[ignore]d (so `--ignored` +# selects them) and install a tool from crates.io. Every test in that file is +# named with the prefix for exactly this. The conformance suite is what this +# target is for. test-metal-device: ## reuse the Metal test graph for on-device conformance @if swift -e 'import Metal; exit(MTLCreateSystemDefaultDevice() == nil ? 1 : 0)' 2>/dev/null; then \ - OXIDE_BACKEND=metal $(CARGO) test $(METAL_TEST_ARGS) -- --ignored; \ + OXIDE_BACKEND=metal $(CARGO) test $(METAL_TEST_ARGS) -- --ignored --skip termlens_cli; \ else \ echo "No Metal device on this runner — on-device conformance skipped."; \ fi @@ -105,13 +118,16 @@ ci-scripts: ## CI change detection, required results, Metal build reuse and stre stress-tui: ## build once and stress all three thread counts (ITERS defaults to 100) python3 .github/scripts/stress-tui.py +skill-version: ## the vendored termlens skill names the version the workspace depends on + ./.github/scripts/check-skill-version.sh + zizmor: ## workflow security audit at the level CI enforces (cargo install --locked zizmor --version 1.29.0) zizmor --persona=pedantic --offline .github/workflows/ deny: ## advisories, licenses, bans and sources (cargo-deny) $(CARGO) deny --all-features check -gate: fmt-check lint lint-cuda lint-predict test test-predict check-cuda check-predict-no-second-cuda doc coherence deny release-scripts crate-metadata ci-scripts zizmor msrv ## the whole quality gate — the same list CI runs +gate: fmt-check lint lint-cuda lint-predict test test-predict check-cuda check-predict-no-second-cuda doc coherence deny release-scripts crate-metadata ci-scripts skill-version zizmor msrv ## the whole quality gate — the same list CI runs ifeq ($(UNAME),Linux) $(MAKE) test-io-uring endif diff --git a/STATUS.md b/STATUS.md index 8d2ff04..d0a76b1 100644 --- a/STATUS.md +++ b/STATUS.md @@ -12,7 +12,7 @@ Legend: `not started` · `in progress` · `done` · `blocked` | 3 | Kernels + oxidelake-compute operators | done | gate green; conformance suite `cargo test -p oxidelake-compute` (4 tests: CPU backend vs stock DataFusion over 3 seeds × 5 sizes × 5 checks, EXPLAIN tags, construction validation); `cargo test -p oxidelake-device --features cuda` 18 passed / 2 ignored | **Kernel execution verified only where a device exists**: the CUDA launch code type-checks under `--features cuda` but no kernel has been NVRTC-compiled or run here; the Metal dispatch is not compiled on Linux at all | | 4 | oxidelake-storage — Parquet pruning config, Arrow IPC spill files, io_uring ObjectStore | done | gate green incl. `cargo test -p oxidelake-storage --features io-uring`; pruning proof: statistics prune ≥90/100 row groups on `k = 42`, Bloom filters prune ≥90/100 on `id = ` with results equal to the pruning-disabled run; io_uring store passes the same behavioural suite as `LocalFileSystem` (executed here — `io_uring_setup` works in this sandbox); truncated/garbage Parquet → `EngineError::Format` | io_uring store: sequential ring worker (one SQE in flight) — correct, not yet tuned; **thrift CVE-2026-43868 via parquet 58 accepted with reason** — fix is parquet 59, blocked on Ballista > 54 (#4) | | 5 | oxidelake-planner (placement + codec), oxidelake-runtime on Ballista | done | gate green; placement tests (mocked GPU target: Filter/Projection fusion, HashJoin and SortMergeJoin lowering with DataFusion's build/probe order and pushed projections, Final/Partial aggregate pairs with DataFusion's output names; ineligible nodes untouched; CPU target = no rewrites); codec round-trips every node; **in-process Ballista scheduler + 2 executors return exactly the embedded results** on 4 queries with `Gpu*Exec` nodes in the scheduler's plans (executors take the CPU path); `oxide`, `oxide-scheduler`, `oxide-worker` print `--help` | Cluster `EXPLAIN` tags are asserted on the plan the scheduler's session builder produces (the client-side plan is Ballista's `DistributedQueryExec`); Ballista's shuffle/retry behaviour untouched | -| 6 | oxidelake-tui — dashboard, TestBackend + termlens tests | done | gate green; `tests/tui_render_test.rs`: `TestBackend` + `insta` snapshots at 80×24 and 120×40, selection/panel/quit transitions, determinism across 4 sizes; `tests/tui_pty_test.rs`: **termlens drives the real `oxidelake-tui-demo` binary in a PTY** — initial screen, `↓` selection, `Tab`, `↓`, SIGWINCH resize to 120×40, `q` exits 0 (3 screen snapshots); 6 `.snap` files committed; no TTY needed | Dashboard observes the local process only; telemetry gauges use fixed reference capacities until backend `MemoryInfo` is wired in | +| 6 | oxidelake-tui — dashboard, TestBackend + termlens tests | done | gate green; `tests/tui_render_test.rs`: `TestBackend` + `insta` snapshots at 80×24 and 120×40, selection/panel/quit transitions, determinism across 4 sizes; `tests/tui_pty_test.rs`: **termlens 0.10 drives the real `oxidelake-tui-demo` binary in a PTY** — initial screen, `↓` selection, `Tab`, `↓`, SIGWINCH resize to 120×40, `q` exits 0 (3 text snapshots + 1 styled), plus the backend-tag colours at every occurrence and `diff` proving `Tab` changes no text and `↓` leaves the right half alone; `tests/emulation.rs` pins `Screen::unsupported() == ["^[[59m"]`, the terminal modes and the snapshot-text/JSON round trips; `tests/termlens_cli.rs` reads the committed screens back through `termlens-cli` (`#[ignore]`d, CI runs it); `crates/oxidelake-runtime/tests/oxide_tui_pty.rs` does the same for the shipped `oxide tui`; 7 `.snap` files committed; no TTY needed | Dashboard observes the local process only; telemetry gauges use fixed reference capacities until backend `MemoryInfo` is wired in | | 7 | oxidelake-api, end-to-end CLI, docs | done | gate green (default + `cuda` + `metal` lanes); UDF tests in `oxidelake-compute` (reference-equality, nulls, coercion errors); vector lowering in `oxidelake-planner/tests/placement.rs` incl. codec round-trip; DataFrame API vs SQL in `oxidelake-api/tests/dataframe.rs`; `oxidelake-runtime/tests/cli.rs` (assert_cmd): `gen-data` verified by an independent `parquet`-crate read, spawned `oxide-scheduler` + `oxide-worker` print byte-identical `--cluster` results, `explain --target` shows tags; `oxidelake-runtime/tests/embedded.rs`: cpu/cuda/metal targets return identical rows; README quickstart commands executed as written (1M rows, embedded + cluster + tui in a PTY) | PyO3 `python` feature deferred (see below); `oxide tui` without `--query` shows the synthetic demo fixture | ## Verification matrix diff --git a/crates/oxidelake-runtime/Cargo.toml b/crates/oxidelake-runtime/Cargo.toml index 173c0d3..dc23caa 100644 --- a/crates/oxidelake-runtime/Cargo.toml +++ b/crates/oxidelake-runtime/Cargo.toml @@ -9,6 +9,14 @@ authors.workspace = true repository.workspace = true homepage.workspace = true documentation = "https://docs.rs/oxidelake-runtime" +# Kept out of the published archive: it compares `oxide tui` against a screen +# snapshot committed in `crates/oxidelake-tui`, which is deliberately a *sibling +# crate's* file — two independent snapshots of the same frame would drift +# together silently, so the cross-crate read is the point. That path does not +# exist once this crate is unpacked on its own, and a published test that +# cannot find its fixture is a failure handed to whoever runs `cargo test` on +# the vendored source. It runs in the workspace, where the file is there. +exclude = ["tests/oxide_tui_pty.rs"] readme.workspace = true keywords.workspace = true categories.workspace = true @@ -51,6 +59,9 @@ arrow.workspace = true assert_cmd.workspace = true parquet.workspace = true tempfile.workspace = true +# `oxide tui` is the dashboard as users install it; assert_cmd cannot see a +# PTY, so the shipped binary's terminal behaviour is tested with termlens. +termlens.workspace = true [[bin]] name = "oxide" diff --git a/crates/oxidelake-runtime/tests/oxide_tui_pty.rs b/crates/oxidelake-runtime/tests/oxide_tui_pty.rs new file mode 100644 index 0000000..38d536a --- /dev/null +++ b/crates/oxidelake-runtime/tests/oxide_tui_pty.rs @@ -0,0 +1,149 @@ +//! `oxide tui` in a real pseudo-terminal — the dashboard as users get it. +//! +//! `crates/oxidelake-tui` tests the dashboard through `oxidelake-tui-demo`, a +//! binary that exists for the tests. What people install is `oxide` +//! (`cargo install oxidelake-runtime`, the release archives, Homebrew), and +//! `oxide tui` with no `--query` renders the same `demo_model()` through the +//! same `run_terminal` — but behind `clap`, `#[tokio::main]` and a +//! `tracing_subscriber` writing to stderr. `tests/cli.rs` drives the other +//! subcommands with `assert_cmd`, which captures pipes and cannot see any of +//! that: in a terminal stdout and stderr are one stream, so a log line lands +//! *on the dashboard*, and a program that never restores the terminal leaves +//! the user's shell in the alternate screen. Only a PTY test can tell. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use std::time::Duration; + +use termlens::{Key, Screen, Terminal}; + +/// The first frame `oxidelake-tui` commits to for the demo model, at 80x24 — +/// the file `oxidelake-tui`'s own PTY suite snapshots, read back with the +/// parser termlens ships for it. +const DEMO_SNAPSHOT: &str = concat!( + env!("CARGO_MANIFEST_DIR"), + "/../oxidelake-tui/tests/snapshots/tui_pty_test__the_first_frame_carries_its_colours_too.snap" +); + +/// `oxide tui`, painted and settled. +fn tui(log: Option<&str>) -> termlens::Result { + let builder = Terminal::builder() + .size(80, 24) + .env_clear() + .timeout(Duration::from_secs(60)) + .args(["tui"]); + let builder = match log { + // The subscriber is installed before `clap` parses, and its writer is + // stderr — which in a PTY is the dashboard's own stream. + Some(level) => builder.env("RUST_LOG", level), + None => builder, + }; + let mut t = builder.spawn(env!("CARGO_BIN_EXE_oxide"))?; + t.wait_frame(|s| s.contains("OxideLake") && s.contains("Plan DAG"))?; + Ok(t) +} + +/// The screen `oxidelake-tui` committed, as a `Screen`. +fn committed_frame() -> Screen { + let raw = std::fs::read_to_string(DEMO_SNAPSHOT).unwrap_or_else(|e| { + panic!( + "{DEMO_SNAPSHOT}: {e} — oxidelake-tui's PTY snapshot is what this test compares against" + ) + }); + let mut lines = raw.lines(); + assert_eq!(lines.next(), Some("---"), "insta writes a header first"); + let body: Vec<&str> = lines.skip_while(|line| *line != "---").skip(1).collect(); + Screen::parse(&format!("{}\n", body.join("\n"))).expect("the committed snapshot parses") +} + +/// The shipped binary draws the dashboard the library's own suite snapshots +/// — cell for cell, colours included — and hands the terminal back. +/// +/// Comparing against `oxidelake-tui`'s committed screen rather than a second +/// snapshot of its own is the whole point: the failure this guards against +/// is `oxide tui` drifting away from the frame the dashboard crate tests, +/// and two independent snapshots would drift together silently. +#[test] +fn oxide_tui_paints_the_dashboard_the_tui_crate_snapshots() -> termlens::Result<()> { + let mut t = tui(None)?; + let live = t.screen(); + assert!( + live.alternate_screen(), + "a full-screen TUI holds the alternate screen while it runs" + ); + + let committed = committed_frame(); + let diff = committed.diff(&live); + assert!( + diff.is_empty(), + "`oxide tui` no longer renders the frame oxidelake-tui snapshots:\n{diff}" + ); + + // The same emulator invariant `oxidelake-tui/tests/emulation.rs` pins, + // asserted again here because this is a different binary: `oxide` links + // the whole engine, and anything it printed would come through the same + // stream. `^[[59m` is ratatui's underline-colour reset; it changes no + // cell. + assert_eq!( + live.unsupported() + .iter() + .map(|u| u.to_string()) + .collect::>(), + ["^[[59m"], + "the shipped binary emitted a sequence termlens does not model:\n{live}" + ); + assert_eq!(live.unsupported_overflow(), 0); + + t.send(Key::Char('q'))?; + let status = t.wait_exit()?; + assert!(status.success(), "`oxide tui` exited with {status:?}"); + let after = t.screen(); + assert!( + !after.alternate_screen(), + "left the user's shell inside the alternate screen" + ); + assert!(after.cursor().2, "left the cursor hidden"); + assert!( + after.text().trim().is_empty() && after.scrollback_rows() == 0, + "`oxide tui` left output behind on the restored terminal:\n{after}" + ); + Ok(()) +} + +/// The log stream shares the terminal with the dashboard, and must stay off +/// the grid. +/// +/// `main` installs a `tracing_subscriber` writing to stderr before anything +/// else happens, and in a PTY stderr *is* the screen: one `info!` on the +/// startup path would paint over a panel, scroll the frame, or leave a line +/// behind after the restore. The demo path opens no session, so at `info` +/// there is nothing to say — and this is how that stays true. If it fails, +/// the fix is to route the subscriber somewhere other than the terminal the +/// dashboard owns, not to lower the level. +#[test] +fn logging_at_info_puts_nothing_on_the_dashboard() -> termlens::Result<()> { + let quiet = tui(None)?.screen(); + let mut t = tui(Some("info"))?; + let noisy = t.screen(); + + let diff = quiet.diff(&noisy); + assert!( + diff.is_empty(), + "RUST_LOG=info changed what `oxide tui` draws — a log line is on the \ + dashboard:\n{diff}" + ); + assert_eq!( + noisy.scrollback_rows(), + 0, + "a log line scrolled the frame:\n{noisy}" + ); + + t.send(Key::Char('q'))?; + assert!(t.wait_exit()?.success()); + let after = t.screen(); + assert!( + after.text().trim().is_empty(), + "a log line survived the restore:\n{after}" + ); + Ok(()) +} diff --git a/crates/oxidelake-tui/Cargo.toml b/crates/oxidelake-tui/Cargo.toml index ffc393d..dd2669d 100644 --- a/crates/oxidelake-tui/Cargo.toml +++ b/crates/oxidelake-tui/Cargo.toml @@ -24,6 +24,7 @@ crossterm.workspace = true [dev-dependencies] insta.workspace = true termlens.workspace = true +serde_json.workspace = true [[bin]] name = "oxidelake-tui-demo" diff --git a/crates/oxidelake-tui/tests/emulation.rs b/crates/oxidelake-tui/tests/emulation.rs new file mode 100644 index 0000000..98f8e7b --- /dev/null +++ b/crates/oxidelake-tui/tests/emulation.rs @@ -0,0 +1,187 @@ +//! What the emulator can and cannot see of the dashboard — the assertion the +//! rest of the PTY suite rests on. +//! +//! Every screen assertion in `tui_pty_test.rs` reads a grid a VT emulator +//! built out of ratatui's bytes. If the application emits a sequence the +//! emulator does not implement, that grid is quietly wrong and the three +//! committed snapshots, the `contains("1000000")` waits and the colour rules +//! are all being made against a plausible-looking fiction. termlens 0.10 made +//! that checkable: `Screen::unsupported` lists what was dropped. +//! +//! These are whole-suite invariants rather than feature tests. They are +//! cheap, and when one breaks the right response is to distrust the other +//! files until it is understood. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use termlens::{Key, Screen, Terminal}; + +/// The only sequence this dashboard emits that termlens does not model. +/// +/// `SGR 59` is "underline colour: default". `ratatui-crossterm` queues +/// `SetUnderlineColor(Reset)` at the tail of every `draw` when its +/// `underline-color` feature is on, whether or not anything is underlined; +/// termlens carries no underline colour, so it records the sequence and +/// moves on. It changes no cell, which is the whole reason this list can be +/// pinned exactly: anything joining it is a sequence that *might* change a +/// cell and has to be read before the suite is trusted again. +/// +/// Note what is *not* here. termlens#320 reports `^[[5m` / `^[[25m` / +/// `^[[9m` / `^[[29m` — blink and strikethrough — as unsupported although +/// the attribute shadow implements them, so those four are false positives +/// wherever they appear. This dashboard neither blinks nor strikes through +/// (`render.rs` uses BOLD, a foreground and a background and nothing else), +/// so they do not appear and this pin needs no exception. +const EXPECTED_UNSUPPORTED: [&str; 1] = ["^[[59m"]; + +fn spawn(cols: u16, rows: u16) -> termlens::Result { + termlens::bin!( + "oxidelake-tui-demo", + size(cols, rows), + timeout(std::time::Duration::from_secs(20)) + ) +} + +fn unsupported(screen: &Screen) -> Vec { + screen.unsupported().iter().map(|s| s.to_string()).collect() +} + +fn check(label: &str, screen: &Screen) { + assert_eq!( + unsupported(screen), + EXPECTED_UNSUPPORTED, + "{label}: the dashboard emitted a sequence termlens does not model. \ + Until it is understood, every screen assertion in this crate is being \ + made against a grid that may be wrong.\n{screen}" + ); + assert_eq!( + screen.unsupported_overflow(), + 0, + "{label}: the record is complete, not truncated" + ); +} + +/// The invariant, at every instant the suite actually asserts against: the +/// first paint, a navigation repaint, a SIGWINCH repaint at another size, +/// and the teardown that hands the terminal back. +/// +/// The resize is the one worth checking hardest — it is the only path that +/// re-enters the alternate screen machinery, and a dropped sequence there +/// would land in the 120x40 snapshot. +#[test] +fn the_emulator_drops_nothing_that_could_change_a_cell() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + check( + "first paint", + &t.wait_frame(|s| s.contains("OxideLake") && s.contains("Describe"))?, + ); + + t.send(Key::Down)?; + check( + "after ↓", + &t.wait_frame(|s| s.contains("1000000") && s.contains("32.0 MiB"))?, + ); + + t.resize(120, 40)?; + check( + "after SIGWINCH", + &t.wait_frame(|s| s.size() == (120, 40) && s.contains("Describe"))?, + ); + + t.send(Key::Char('q'))?; + assert!(t.wait_exit()?.success()); + check("after quit", &t.screen()); + Ok(()) +} + +/// Four smaller invariants that would each make the grid a lie, and that +/// nothing else in the suite would notice. +#[test] +fn the_dashboard_leaves_the_terminal_modes_alone() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + let screen = t.wait_frame(|s| s.contains("OxideLake") && s.contains("Describe"))?; + + // Insert mode pushes the rest of a row right. An application that left + // it on would draw a correct-looking dashboard with every row shifted, + // and the box drawing would still line up. + assert!(!screen.insert_mode(), "the dashboard never sets IRM"); + // A visual bell is a flash the grid cannot show. + assert_eq!(screen.visual_bells(), 0, "no ESC g"); + assert_eq!(screen.bells(), 0, "and no audible bell either"); + // Nothing wraps: ratatui lays out to the width it was given, so a + // wrapped row means a panel overflowed and its text is silently on two + // rows — which reads as a correct dashboard one row short. + assert!( + !(0..screen.rows()).any(|row| screen.row_wrapped(row)), + "a wrapped row means the layout overflowed:\n{screen}" + ); + assert_eq!( + screen.logical_text(), + screen.text(), + "with no wrapped row the logical text is the grid text" + ); + // The dashboard enables no mouse tracking, so `click`/`drag`/`scroll` + // have nothing to reach here — asserted rather than left as folklore, + // because "the mouse does not work" is otherwise indistinguishable from + // "nobody wrote the test". + assert!( + screen.mouse_modes().is_empty(), + "no mouse mode is enabled: {:?}", + screen.mouse_modes() + ); + + // And the shell gets its terminal back with nothing left in it: the app + // holds the alternate screen for its whole life, so a line printed + // outside it (a panic message, a stray println) would survive the + // restore and show up here. + t.send(Key::Char('q'))?; + assert!(t.wait_exit()?.success()); + let after = t.screen(); + assert!(!after.alternate_screen()); + assert_eq!(after.scrollback_rows(), 0, "nothing scrolled into history"); + assert!( + after.text().trim().is_empty(), + "the restored screen is not blank:\n{after}" + ); + Ok(()) +} + +/// A dashboard screen has to survive being saved and read back, because that +/// is what a bug report and a CI artefact are: the box drawing, the block +/// glyphs and every colour come back, or the format is not carrying what +/// this application draws. +#[test] +fn a_dashboard_screen_survives_the_snapshot_format_and_json() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + let screen = t.wait_frame(|s| s.contains("OxideLake") && s.contains("[CUDA]"))?; + + // The text format: a committed `.snap`, the block a wait error prints, + // and what `termlens diff` reads back (see `termlens_cli.rs`). + let saved = screen.with_styles().to_string(); + let parsed = Screen::parse(&saved)?; + assert!(screen.diff(&parsed).is_empty(), "{}", screen.diff(&parsed)); + assert_eq!(parsed.with_styles().to_string(), saved, "byte for byte"); + + // And JSON, which is what `TERMLENS_ARTIFACT_DIR` writes in CI — the + // file the report action renders when a wait fails on a runner. + let json = serde_json::to_string(&screen).expect("a Screen serializes"); + let back: Screen = serde_json::from_str(&json).expect("and comes back"); + assert!(screen.diff(&back).is_empty(), "{}", screen.diff(&back)); + + // The colours specifically: the panels encode the backend and the focus + // in colour alone, so a round trip that dropped styles would still pass + // a text comparison. + let (row, col) = screen.find("[CUDA]").expect("the plan tags its operators"); + let cell = screen.cell(row, col).unwrap(); + assert_ne!(cell.style().fg, termlens::Color::Default, "measured green"); + for (label, other) in [("the text format", &parsed), ("JSON", &back)] { + assert_eq!( + other.cell(row, col).unwrap().style(), + cell.style(), + "{label} lost the colour of the [CUDA] tag at ({row},{col})" + ); + } + t.send(Key::Char('q'))?; + assert!(t.wait_exit()?.success()); + Ok(()) +} diff --git a/crates/oxidelake-tui/tests/snapshots/tui_pty_test__the_first_frame_carries_its_colours_too.snap b/crates/oxidelake-tui/tests/snapshots/tui_pty_test__the_first_frame_carries_its_colours_too.snap new file mode 100644 index 0000000..ece3766 --- /dev/null +++ b/crates/oxidelake-tui/tests/snapshots/tui_pty_test__the_first_frame_carries_its_colours_too.snap @@ -0,0 +1,46 @@ +--- +source: crates/oxidelake-tui/tests/tui_pty_test.rs +expression: __screen.with_styles() +--- +size: 80x24 cursor: hidden +OxideLake data/t.parquet — SELECT k, sum(v), count(v) FROM t WHERE k >= 2 AND v +┌ Plan DAG ────────────────────────────┐┌ Telemetry ───────────────────────────┐ +│▶ GpuAggregateExec [CUDA] ││█████████████████████████████ │ +│ └─ GpuFilterExec [CUDA] ││███████VRAM: 6.0 GiB / 8.0 GiB │ +│ └─ DataSourceExec [CPU] ││██████████████ │ +│ ││████pinned RAM: 1.5 GiB / 4.0 GiB │ +│ ││██████████ │ +│ ││███disk spill: 512.0 MiB / 2.0 GiB │ +│ ││PCIe H2D 35.3 MiB D2H 18.6 MiB │ +│ ││NVMe spilled 3.0 GiB (42 demotions) │ +│ │└──────────────────────────────────────┘ +│ │┌ Describe ────────────────────────────┐ +└──────────────────────────────────────┘│colu type min max nul p25 p50 p99 │ +┌ Inspector ───────────────────────────┐│k Int6 0 99 0 24 49 98 │ +│GpuAggregateExec [CUDA] ││v Floa 0.0 24.0 769 6.0 12.0 23.7│ +│group_by=[k], aggr=[SUM(v), COUNT(v)] ││s Utf8 s0 s6 0 │ +│ ││ │ +│rows in 812400 ││ │ +│rows out 9 ││ │ +│batches 100 ││ │ +│latency/batch 1.840 ms ││ │ +│memory 16.0 MiB ││ │ +└──────────────────────────────────────┘└──────────────────────────────────────┘ + ↑/↓ select operator Tab next panel q quit + +styles: +0: 0-8 fg=6 bold +1: 0-39 fg=3 bold +2: 0 fg=3 bold; 1-19 bg=8 bold; 20-25 fg=2 bg=8 bold; 26-38 bg=8 bold; 39 fg=3 bold; 41-78 fg=2 +3: 0 fg=3 bold; 8-20 bold; 22-27 fg=2 bold; 39 fg=3 bold; 41-47 fg=2; 48-69 bg=2; 70-78 fg=2 +4: 0 fg=3 bold; 10-23 bold; 25-29 fg=4 bold; 39 fg=3 bold; 41-78 fg=6 +5: 0 fg=3 bold; 39 fg=3 bold; 41-44 fg=6; 45-54 bg=6; 55-78 fg=6 +6: 0 fg=3 bold; 39 fg=3 bold; 41-78 fg=3 +7: 0 fg=3 bold; 39 fg=3 bold; 41-43 fg=3; 44-50 bg=3; 51-78 fg=3 +8: 0 fg=3 bold; 39 fg=3 bold +9: 0 fg=3 bold; 39 fg=3 bold +10: 0 fg=3 bold; 39 fg=3 bold +11: 0 fg=3 bold; 39 fg=3 bold +12: 0-39 fg=3 bold; 41-78 bold +14: 1-16 bold; 18-23 fg=2 bold +23: 0-4 bold; 22-24 bold; 38 bold diff --git a/crates/oxidelake-tui/tests/termlens_cli.rs b/crates/oxidelake-tui/tests/termlens_cli.rs new file mode 100644 index 0000000..18b5c4f --- /dev/null +++ b/crates/oxidelake-tui/tests/termlens_cli.rs @@ -0,0 +1,292 @@ +//! `termlens-cli` — the harness this suite already uses, at a shell prompt. +//! +//! The rest of the suite asks whether the dashboard draws the right thing. +//! This asks what a maintainer does *after* it draws the wrong thing: point +//! the tool at the real binary, read a committed `.snap` back, and diff two +//! of them to see what a keystroke changed. Those `.snap` files are the +//! repository's own record of what OxideLake looks like, and nothing +//! verified that anything except `cargo insta` could still read them. +//! +//! Every test here is `#[ignore]`d. They need `termlens-cli` on the machine, +//! and these crates are published — a `cargo test` that quietly +//! `cargo install`s something is a surprise a contributor should not get. +//! CI runs them by name (`make test-termlens-cli`), and the version it +//! installs is the one `Cargo.lock` names, so the tool and the library can +//! never be two different releases. +//! +//! ```sh +//! cargo test -p oxidelake-tui --test termlens_cli -- --ignored +//! TERMLENS_CLI=$(command -v termlens) cargo test -p oxidelake-tui --test termlens_cli -- --ignored +//! ``` +//! +//! Every test name starts with `termlens_cli_` on purpose: the macOS Metal +//! lane runs the whole `-p oxidelake-tui` graph a second time with +//! `-- --ignored` for the on-device conformance suite, and skips these by +//! that prefix rather than installing a tool from the network mid-gate. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; +use std::sync::OnceLock; + +use termlens::Screen; + +/// The three screens this repository has committed, as `termlens diff` and +/// `termlens render` see them: an insta `.snap` is the snapshot text format +/// with a header, which the tool reads. +const SNAPSHOTS: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/tests/snapshots/"); +/// The first frame, text only. +const FIRST: &str = "tui_pty_test__demo_renders_navigates_resizes_and_quits.snap"; +/// The same frame after `↓` — so a diff of the two is what `↓` did. +const AFTER_DOWN: &str = "tui_pty_test__demo_renders_navigates_resizes_and_quits-2.snap"; +/// The first frame again, with its `styles:` block. +const STYLED: &str = "tui_pty_test__the_first_frame_carries_its_colours_too.snap"; + +fn snapshot(name: &str) -> PathBuf { + PathBuf::from(SNAPSHOTS).join(name) +} + +/// The termlens version this suite is measured against, read from the +/// lockfile so the tool and the library can never be two different releases. +fn version_under_test() -> &'static str { + static VERSION: OnceLock = OnceLock::new(); + VERSION.get_or_init(|| { + let lock = + std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/../../Cargo.lock")) + .expect("Cargo.lock is committed at the workspace root"); + let mut lines = lock.lines(); + while let Some(line) = lines.next() { + if line.trim() == "name = \"termlens\"" { + for next in lines.by_ref() { + if let Some(rest) = next.trim().strip_prefix("version = \"") { + return rest.trim_end_matches('"').to_owned(); + } + } + } + } + panic!("no termlens version in Cargo.lock"); + }) +} + +/// The workspace `target/`, which is where an installed tool belongs: not +/// `crates/oxidelake-tui/target/`, which nothing else in this workspace uses +/// and `make clean` would miss. +fn target_dir() -> PathBuf { + if let Some(dir) = std::env::var_os("CARGO_TARGET_DIR") { + return PathBuf::from(dir); + } + // The test binary is //deps/. + std::env::current_exe() + .ok() + .and_then(|exe| exe.ancestors().nth(3).map(Path::to_path_buf)) + .unwrap_or_else(|| PathBuf::from(concat!(env!("CARGO_MANIFEST_DIR"), "/../../target"))) +} + +/// The `termlens` binary: `$TERMLENS_CLI` if the environment provides one, +/// otherwise installed once into `target/` at the version under test. +fn cli() -> &'static PathBuf { + static BIN: OnceLock = OnceLock::new(); + BIN.get_or_init(|| { + if let Some(given) = std::env::var_os("TERMLENS_CLI") { + return PathBuf::from(given); + } + let root = target_dir().join("termlens-cli"); + let bin = root + .join("bin") + .join(format!("termlens{}", std::env::consts::EXE_SUFFIX)); + let cargo = std::env::var_os("CARGO").unwrap_or_else(|| "cargo".into()); + let status = Command::new(cargo) + .args(["install", "termlens-cli", "--version", version_under_test()]) + .args(["--locked", "--root"]) + .arg(&root) + .status() + .expect("cargo install termlens-cli"); + assert!( + status.success(), + "cargo install termlens-cli --version {} failed. It is published \ + alongside the library; if this version of termlens exists on \ + crates.io and termlens-cli does not, the two releases went out \ + of lockstep.", + version_under_test() + ); + bin + }) +} + +fn run(args: &[&str]) -> Output { + Command::new(cli()) + .args(args) + .output() + .expect("run termlens") +} + +fn stdout(out: &Output) -> String { + String::from_utf8_lossy(&out.stdout).into_owned() +} + +/// A `.snap` without insta's `---` header: the screen the file records. +fn snapshot_body(name: &str) -> String { + let raw = std::fs::read_to_string(snapshot(name)).expect("a committed snapshot"); + let mut lines = raw.lines(); + assert_eq!(lines.next(), Some("---"), "insta writes a header first"); + let body: Vec<&str> = lines.skip_while(|line| *line != "---").skip(1).collect(); + format!("{}\n", body.join("\n")) +} + +#[test] +#[ignore = "needs termlens-cli; run with --ignored (CI does)"] +fn termlens_cli_is_the_release_the_library_comes_from() { + let out = run(&["--version"]); + assert!(out.status.success()); + assert_eq!( + stdout(&out).trim(), + format!("termlens {}", version_under_test()), + "the installed CLI is not the version this suite tests against" + ); +} + +/// `inspect` points the harness at a binary without writing a test, which is +/// the first thing a contributor reaches for — and the first thing the +/// README of a TUI should be able to promise. Pointed at the dashboard it +/// has to survive raw mode, the alternate screen and DEC 2026 frames. +#[test] +#[ignore = "needs termlens-cli; run with --ignored (CI does)"] +fn termlens_cli_inspect_drives_the_real_dashboard() { + let out = Command::new(cli()) + .args(["inspect", "--size", "100x30", "--idle", "600"]) + .arg(env!("CARGO_BIN_EXE_oxidelake-tui-demo")) + .output() + .expect("run inspect"); + assert!( + out.status.success(), + "{}", + String::from_utf8_lossy(&out.stderr) + ); + let screen = stdout(&out); + assert!(screen.contains("size: 100x30"), "{screen}"); + for panel in ["Plan DAG", "Inspector", "Telemetry", "Describe"] { + assert!(screen.contains(panel), "no {panel} panel:\n{screen}"); + } + assert!(screen.contains("[CUDA]"), "the plan tags its operators"); + assert!( + screen.contains("still running at the deadline"), + "the dashboard is a TUI, so inspect reports the deadline rather than \ + an exit — a binary that fell out of the event loop would say so here:\n{screen}" + ); +} + +/// The committed screens are still readable, and still carry their colours. +/// +/// `render --text` is the round trip: what the tool prints must be the file, +/// exactly. The image renderings are what a bug report or a step summary +/// shows, and for this dashboard the colour *is* the information — a green +/// `[CUDA]` beside a blue `[CPU]` is how a reader sees where an operator +/// ran. +#[test] +#[ignore = "needs termlens-cli; run with --ignored (CI does)"] +fn termlens_cli_renders_the_committed_screens_with_their_colours() { + let styled = snapshot(STYLED); + let path = styled.to_str().unwrap(); + + let text = run(&["render", "--text", path]); + assert!(text.status.success()); + assert_eq!( + stdout(&text), + snapshot_body(STYLED), + "`termlens render --text` no longer reproduces the committed screen" + ); + + // What the screen itself says the tags are drawn in, read with the same + // parser the CLI uses, so the expectation below is measured rather than + // remembered. + let screen = Screen::parse(&snapshot_body(STYLED)).expect("the snapshot parses"); + let (row, col) = screen.find("[CUDA]").expect("a tagged operator"); + assert_eq!( + screen.cell(row, col).unwrap().style().fg, + termlens::Color::Indexed(2), + "the CUDA tag is palette green (render.rs: Color::Green)" + ); + + // `#0dbc79` and `#2472c8` are how termlens renders palette entries 2 and + // 4 into an image. Tying the colour to the text in one assertion is the + // point: an SVG with the right words in the wrong colours is exactly the + // regression a text snapshot cannot see. + let svg = run(&["render", "--svg", path]); + assert!(svg.status.success()); + let body = stdout(&svg); + assert_eq!( + body.matches(r##"fill="#0dbc79" xml:space="preserve" font-weight="bold">[CUDA]"##) + .count(), + screen.find_all("[CUDA]").len(), + "every [CUDA] tag should be green in the image:\n{body}" + ); + assert_eq!( + body.matches(r##"fill="#2472c8" xml:space="preserve" font-weight="bold">[CPU]"##) + .count(), + screen.find_all("[CPU]").len(), + "and every [CPU] tag blue:\n{body}" + ); + + let html = run(&["render", "--html", path]); + assert!(html.status.success()); + let body = stdout(&html); + assert!(body.contains("color:#0dbc79"), "the HTML keeps the palette"); + assert!(body.contains("[CUDA]"), "and the text:\n{body}"); +} + +/// The workflow a maintainer runs when a frame changed and they want to know +/// exactly how — on this repository's own files. The three exit codes are +/// the part a script reads: 0 same, 1 different, 2 could not be read. +#[test] +#[ignore = "needs termlens-cli; run with --ignored (CI does)"] +fn termlens_cli_diffs_this_repositorys_own_snapshots() { + let first = snapshot(FIRST); + let first = first.to_str().unwrap(); + let after_down = snapshot(AFTER_DOWN); + let after_down = after_down.to_str().unwrap(); + + let same = run(&["diff", "--color", "never", first, first]); + assert_eq!(same.status.code(), Some(0), "a screen equals itself"); + assert!(stdout(&same).contains("no difference")); + + // These two files are the frames either side of `↓` in + // `demo_renders_navigates_resizes_and_quits`, so the diff is that + // keystroke, spelled out. + let moved = run(&["diff", "--color", "never", first, after_down]); + assert_eq!( + moved.status.code(), + Some(1), + "↓ selects another operator, so the two frames differ" + ); + let rendered = stdout(&moved); + assert!(rendered.contains("size: 80x24"), "the header:\n{rendered}"); + assert!( + rendered.contains("rows unchanged"), + "and a count of what did not move:\n{rendered}" + ); + assert!( + rendered.contains("1000000"), + "the Inspector's new row count is what ↓ produced:\n{rendered}" + ); + + // Exit 2 is the tool failing, not a difference: a script that treats + // "could not read the file" as "the screens differ" reports a phantom + // regression on every typo. + let not_a_screen = run(&[ + "diff", + "--color", + "never", + first, + concat!(env!("CARGO_MANIFEST_DIR"), "/Cargo.toml"), + ]); + assert_eq!( + not_a_screen.status.code(), + Some(2), + "a manifest is not a screen" + ); + assert!( + String::from_utf8_lossy(¬_a_screen.stderr).contains("could not parse a saved screen"), + "and it says so" + ); +} diff --git a/crates/oxidelake-tui/tests/tui_pty_test.rs b/crates/oxidelake-tui/tests/tui_pty_test.rs index 3e5e773..75b58c3 100644 --- a/crates/oxidelake-tui/tests/tui_pty_test.rs +++ b/crates/oxidelake-tui/tests/tui_pty_test.rs @@ -14,6 +14,15 @@ //! Never a sleep. Every test also asserts the terminal was given back — an //! application that leaves the user in the alternate screen with a hidden //! cursor is a bug only a real PTY can see. +//! +//! **Text and colour are snapshotted separately, on purpose.** termlens 0.10 +//! records styles by default; the three flow snapshots below stay text-only +//! (`styles = false`) so a colour change shows up in *one* place — +//! `the_first_frame_carries_its_colours_too` — instead of rewriting three +//! large files at once. The claims colour actually encodes (the focused +//! panel, the backend of each operator) are asserted directly against cells +//! further down, where a failure names the rule that broke rather than +//! handing over a grid to read. #![allow(clippy::unwrap_used, clippy::expect_used)] @@ -64,12 +73,12 @@ fn demo_renders_navigates_resizes_and_quits() -> termlens::Result<()> { first.alternate_screen(), "a full-screen TUI should hold the alternate screen while it runs" ); - termlens::assert_screen_snapshot!(first); + termlens::assert_screen_snapshot!(&first, styles = false); // ↓ selects GpuFilterExec: the inspector shows its 1,000,000 input rows. t.send(Key::Down)?; let selected = t.wait_frame(|s| s.contains("1000000") && s.contains("32.0 MiB"))?; - termlens::assert_screen_snapshot!(selected); + termlens::assert_screen_snapshot!(&selected, styles = false); // Tab moves the focus highlight; ↓ selects DataSourceExec (8.0 MiB). t.send(Key::Tab)?; @@ -80,7 +89,7 @@ fn demo_renders_navigates_resizes_and_quits() -> termlens::Result<()> { t.resize(120, 40)?; let resized = t.wait_frame(|s| s.size() == (120, 40) && s.contains("Describe") && s.contains("8.0 MiB"))?; - termlens::assert_screen_snapshot!(resized); + termlens::assert_screen_snapshot!(&resized, styles = false); quit(&mut t) } @@ -108,3 +117,210 @@ fn the_app_really_emits_synchronized_frames() -> termlens::Result<()> { ); quit(&mut t) } + +/// The same first frame, with every style it is drawn in. +/// +/// The three snapshots above are text; this is the only place in the +/// repository where the dashboard's colours are recorded wholesale — +/// `TestBackend::to_string()` in `tui_render_test.rs` cannot see them at +/// all. Its own name, so accepting a colour change never touches the text +/// snapshots and vice versa. +#[test] +fn the_first_frame_carries_its_colours_too() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + let first = t.wait_frame(|s| { + s.contains("OxideLake") && s.contains("Plan DAG") && s.contains("Describe") + })?; + termlens::assert_screen_snapshot!(&first); + quit(&mut t) +} + +/// Yellow index in the 256-colour palette: `Color::Yellow` through crossterm. +const YELLOW: termlens::Color = termlens::Color::Indexed(3); +/// Green: what `backend_tag` paints a CUDA operator (`render.rs`). +const GREEN: termlens::Color = termlens::Color::Indexed(2); +/// Blue: the same function's CPU tag. +const BLUE: termlens::Color = termlens::Color::Indexed(4); + +/// The title of `panel` as it is drawn on this screen. +fn border_title(screen: &Screen, panel: &str) -> termlens::Style { + let (row, col) = screen + .find(panel) + .unwrap_or_else(|| panic!("no `{panel}` border on:\n{screen}")); + *screen + .cell(row, col) + .expect("the cell find just reported") + .style() +} + +/// Every occurrence of `tag`, and the style of each of its characters. +fn tag_styles(screen: &Screen, tag: &str) -> Vec<(u16, u16, termlens::Style)> { + screen + .find_all(tag) + .into_iter() + .flat_map(|(row, col)| { + (0..tag.chars().count() as u16).map(move |offset| (row, col + offset)) + }) + .map(|(row, col)| { + ( + row, + col, + *screen + .cell(row, col) + .unwrap_or_else(|| panic!("({row},{col}) is off the grid")) + .style(), + ) + }) + .collect() +} + +/// The placement rule, as a rule rather than as a picture. +/// +/// `render.rs` paints `[CUDA]` green and `[CPU]` blue wherever an operator +/// is shown — twice in the Plan DAG, once again in the Inspector — and that +/// is the one thing on this dashboard a user reads at a glance. A snapshot +/// would pin those three positions; this pins the *rule*, so moving the +/// panels around keeps it true and mixing the two colours up does not. +/// +/// `find_all` is the reason it can be written at all: 0.9 could only find +/// the first occurrence, and the Inspector's copy is the third. +#[test] +fn every_backend_tag_is_drawn_in_the_colour_of_its_backend() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + let first = t.wait_frame(|s| s.contains("[CUDA]") && s.contains("[CPU]"))?; + + // Two GPU operators in the plan and the selected one in the Inspector. + let cuda = tag_styles(&first, "[CUDA]"); + assert_eq!( + first.find_all("[CUDA]").len(), + 3, + "the demo plan has two CUDA operators, and the Inspector repeats the \ + selected one:\n{first}" + ); + for (row, col, style) in &cuda { + assert_eq!(style.fg, GREEN, "[CUDA] at ({row},{col}) is not green"); + assert!(style.bold, "[CUDA] at ({row},{col}) is not bold"); + } + for (row, col, style) in tag_styles(&first, "[CPU]") { + assert_eq!(style.fg, BLUE, "[CPU] at ({row},{col}) is not blue"); + assert!(style.bold, "[CPU] at ({row},{col}) is not bold"); + } + assert!( + first.find_all("[Metal]").is_empty(), + "the demo model has no Metal operator, so nothing should be magenta" + ); + + // And the rule survives the layout moving: ↓ re-renders the Inspector + // around a different operator, so the third tag lands elsewhere. + t.send(Key::Down)?; + let selected = t.wait_frame(|s| s.contains("1000000") && s.contains("32.0 MiB"))?; + assert_ne!( + selected.find_all("[CUDA]"), + first.find_all("[CUDA]"), + "↓ should have moved the Inspector's copy of the tag" + ); + for (row, col, style) in tag_styles(&selected, "[CUDA]") { + assert_eq!(style.fg, GREEN, "[CUDA] at ({row},{col}) lost its colour"); + } + for (row, col, style) in tag_styles(&selected, "[CPU]") { + assert_eq!(style.fg, BLUE, "[CPU] at ({row},{col}) lost its colour"); + } + quit(&mut t) +} + +/// Tab moves the focus, and the focus is *only* a colour. +/// +/// Nothing else in this repository can state this: the border characters are +/// identical either way, so the text snapshots and every `TestBackend` +/// snapshot are byte-for-byte the same before and after. `diff` sees it +/// because it compares cells, styles included — so the assertion is that the +/// text did not move and the picture still changed. +#[test] +fn tab_moves_the_focus_highlight_and_changes_no_text() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + let before = t.wait_frame(|s| s.contains("Plan DAG") && s.contains("Inspector"))?; + assert_eq!( + border_title(&before, "Plan DAG").fg, + YELLOW, + "the Plan DAG panel starts focused" + ); + assert_eq!( + border_title(&before, "Inspector").fg, + termlens::Color::Default + ); + + t.send(Key::Tab)?; + let after = t.wait_frame(|s| border_title(s, "Inspector").fg == YELLOW)?; + assert_eq!( + border_title(&after, "Plan DAG").fg, + termlens::Color::Default, + "the highlight moved off Plan DAG rather than being drawn twice" + ); + + assert_eq!( + before.text(), + after.text(), + "Tab changed the text; it should only change which border is yellow" + ); + let diff = before.diff(&after); + assert!( + !diff.is_empty(), + "Tab changed nothing at all — the focus highlight is gone:\n{after}" + ); + // Plan DAG and Inspector are the two panels stacked in the left half, so + // a repaint of the right half means the focus move dragged the telemetry + // gauges and the Describe table along with it. + let stray: Vec<(u16, u16)> = diff + .cells() + .map(|(row, col, ..)| (row, col)) + .filter(|(_, col)| *col >= 40) + .collect(); + assert!( + stray.is_empty(), + "Tab repainted the right half at {stray:?}:\n{diff}" + ); + quit(&mut t) +} + +/// ↓ selects the next operator, and touches nothing it has no business +/// touching. +/// +/// The selection lives in the Plan DAG and the Inspector — both in the left +/// half — so the gauges and the Describe table must come through a keystroke +/// untouched. Nothing forbade a full repaint before this; a plan panel that +/// redrew the whole screen on every arrow key would pass every other test +/// here and flicker on a real terminal. +#[test] +fn moving_the_selection_leaves_the_right_half_alone() -> termlens::Result<()> { + let mut t = spawn(80, 24)?; + let before = t.wait_frame(|s| s.contains("16.0 MiB"))?; + t.send(Key::Down)?; + let after = t.wait_frame(|s| s.contains("1000000") && s.contains("32.0 MiB"))?; + + let diff = before.diff(&after); + assert!(!diff.is_empty(), "↓ selected nothing:\n{after}"); + let stray: Vec<(u16, u16)> = diff + .cells() + .map(|(row, col, ..)| (row, col)) + .filter(|(_, col)| *col >= 40) + .collect(); + assert!( + stray.is_empty(), + "↓ repainted the Telemetry / Describe half at {stray:?}:\n{diff}" + ); + // Said the other way round, on the text rather than the cells. + assert_eq!( + before.rect_text(40.., ..), + after.rect_text(40.., ..), + "the right half of the dashboard changed when the selection moved" + ); + // The rows it did touch are the two panels that show a selection. + let mut rows: Vec = diff.cells().map(|(row, ..)| row).collect(); + rows.dedup(); + assert!( + rows.iter() + .all(|row| (2..=3).contains(row) || (14..=21).contains(row)), + "↓ should repaint the plan rows and the Inspector body, got {rows:?}:\n{diff}" + ); + quit(&mut t) +} diff --git a/docs/SPEC.md b/docs/SPEC.md index aa4eafc..8fe1a6a 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -141,10 +141,10 @@ There is no private file format (ADR-0014). `oxidelake-storage` owns configurati - `TelemetryHub` (in `oxidelake-core`): atomic counters/gauges written by the engine — per-operator rows and latency, bytes per memory tier, transfer and spill rates, plan summary — with cheap consistent snapshot reads. It is the *only* coupling between engine and TUI, which makes the TUI testable with synthetic snapshots. In v1 the TUI observes the local process only; cluster-wide telemetry is out of scope. - The app is a state machine decoupled from rendering: `fn render(state: &AppState, frame: &mut Frame)` is pure; `fn on_event(state: &mut AppState, ev: Event) -> Transition` handles input. Bracket every repaint in DEC 2026 synchronized updates (crossterm `BeginSynchronizedUpdate`/`EndSynchronizedUpdate`) so PTY tests observe complete frames only. - Panels: (1) **Plan DAG** — tree of physical operators with a hardware tag per node `[CUDA]`/`[Metal]`/`[CPU]`; (2) **Inspector** — selected operator's rows in/out, batch latency (ms), memory allocated; (3) **Telemetry** — gauges for VRAM / pinned RAM / disk spill and PCIe/NVMe transfer rates; (4) **Describe** — per-column min, max, null count, quantiles P25/P50/P99. Keys: `↑`/`↓` select DAG node, `Tab` cycles panels, `q` quits. -- A deterministic demo binary `oxidelake-tui-demo` (`[[bin]]` inside `oxidelake-tui`) renders the dashboard from a fixed synthetic telemetry snapshot — no clock, no animation — so PTY tests can spawn it via `env!("CARGO_BIN_EXE_oxide-tui-demo")`. +- A deterministic demo binary `oxidelake-tui-demo` (`[[bin]]` inside `oxidelake-tui`) renders the dashboard from a fixed synthetic telemetry snapshot — no clock, no animation — so PTY tests can spawn it via `env!("CARGO_BIN_EXE_oxidelake-tui-demo")` (`termlens::bin!`). - **Headless tests, two layers:** 1. `crates/oxidelake-tui/tests/tui_render_test.rs` — in-process: `ratatui::backend::TestBackend` + `insta::assert_snapshot!` of the rendered buffer at 80×24 and 120×40; state transitions (`↓`/`↑` selection, `Tab`, `q` → quit) asserted on `AppState`. - 2. `crates/oxidelake-tui/tests/tui_pty_test.rs` — end-to-end with **`termlens`** (dev-dependency; its default feature enables `insta` integration): `Terminal::builder().size(80, 24).env_clear().timeout(..).spawn(env!("CARGO_BIN_EXE_oxide-tui-demo"))`, then `wait_until(|s| s.contains("OxideLake"))` or `wait_frame(..)`, `termlens::assert_screen_snapshot!(t.screen())`, `send(Key::Down)` and re-snapshot, `resize(120, 40)` and re-snapshot, `send(Key::Char('q'))` then `wait_exit()?.success()`. Read https://docs.rs/termlens/0.6.1 before writing these — the API is newer than your training data. Never `sleep`; always use the `wait_*` methods. termlens is a real-PTY harness, so no physical TTY is needed. + 2. `crates/oxidelake-tui/tests/tui_pty_test.rs` — end-to-end with **`termlens`** (dev-dependency; its default feature enables `insta` integration): `termlens::bin!("oxidelake-tui-demo", size(80, 24), timeout(..))`, then `wait_frame(|s| s.contains("OxideLake"))`, `termlens::assert_screen_snapshot!(&first, styles = false)`, `send(Key::Down)` and re-snapshot, `resize(120, 40)` and re-snapshot, `send(Key::Char('q'))` then `wait_exit()?.success()`. Beside it, `tests/emulation.rs` pins what the emulator dropped (`Screen::unsupported`), that no terminal mode was left set and that a screen survives the snapshot text format and JSON; `tests/termlens_cli.rs` drives the committed screens through `termlens-cli` (`#[ignore]`d — it installs the tool); and `crates/oxidelake-runtime/tests/oxide_tui_pty.rs` does the same for the shipped `oxide tui`. Read before writing these — the API is newer than your training data, and the vendored `.claude/skills/termlens/SKILL.md` is the cheat sheet for the version this workspace depends on. Never `sleep`; always use the `wait_*` methods. termlens is a real-PTY harness, so no physical TTY is needed. ### 2.8 API — `oxidelake-api` @@ -204,7 +204,7 @@ Rules: 1. Every external dependency is declared once in `[workspace.dependencies]`; members use `{ workspace = true }`. 2. **Version coherence has one root: Ballista.** Its docs say "Make sure the version of `datafusion` is the same as `ballista`'s!" — so `ballista*` fixes the `datafusion` / `datafusion-proto` major, DataFusion fixes the `arrow` / `parquet` / `object_store` majors, and `arrow-flight` / `tonic` / `prost` arrive transitively through Ballista (not declared directly). Never depend on other `datafusion-*` sub-crates; `datafusion-proto` is the single exception (needed for the plan codec) and is pinned to the identical version (ADR-0009). Gate: `cargo tree --workspace -d -e normal` shows **no duplicate `arrow-*`, `parquet`, `datafusion*`, `object_store`, `tonic` or `prost` majors**. -3. Evidence for the set below (live registry): ballista / ballista-core / ballista-scheduler / ballista-executor 54.1.0 (2026-08-09) require `datafusion ^54`, `datafusion-proto ^54`, `arrow-flight ^58.3`, `object_store ^0.13.2`, `tonic ^0.14`, `prost ^0.14`; datafusion 54.1.0 requires `arrow ^58.3`, `parquet ^58.3`, `object_store ^0.13.2`, `tokio ^1.52`; arrow-flight 58.4 requires `tonic` / `prost ^0.14.1`; ratatui 0.30.2 pairs with crossterm 0.29 through `crossterm_0_29`; cudarc 0.19.9 ships `dynamic-loading`, `nvrtc` and `cuda-*`; objc2-metal 0.3 requires objc2 0.6 and objc2-foundation 0.3; termlens 0.6.1 defaults to `insta`. MSRVs: Ballista, DataFusion 54 and ratatui 1.88 → `rust-version = "1.88"`. DataFusion 55 exists but Ballista is on 54 — Ballista wins. When Ballista moves to a new DataFusion major, re-derive the whole chain; never bump one crate alone. +3. Evidence for the set below (live registry): ballista / ballista-core / ballista-scheduler / ballista-executor 54.1.0 (2026-08-09) require `datafusion ^54`, `datafusion-proto ^54`, `arrow-flight ^58.3`, `object_store ^0.13.2`, `tonic ^0.14`, `prost ^0.14`; datafusion 54.1.0 requires `arrow ^58.3`, `parquet ^58.3`, `object_store ^0.13.2`, `tokio ^1.52`; arrow-flight 58.4 requires `tonic` / `prost ^0.14.1`; ratatui 0.30.2 pairs with crossterm 0.29 through `crossterm_0_29`; cudarc 0.19.9 ships `dynamic-loading`, `nvrtc` and `cuda-*`; objc2-metal 0.3 requires objc2 0.6 and objc2-foundation 0.3; termlens 0.10.1 defaults to `insta` and keeps `serde` / `regex` behind features (MSRV 1.85, far under this workspace's floor). MSRVs: Ballista, DataFusion 54 and ratatui 1.88 → `rust-version = "1.88"`. DataFusion 55 exists but Ballista is on 54 — Ballista wins. When Ballista moves to a new DataFusion major, re-derive the whole chain; never bump one crate alone. ```toml [workspace] @@ -257,10 +257,11 @@ crossterm = "0.29" clap = { version = "4", features = ["derive"] } # test-only insta = "1" -termlens = "0.6" +termlens = { version = "0.10", features = ["serde"] } assert_cmd = "2" tempfile = "3" rand = "0.10" +serde_json = "1" [profile.release] opt-level = 3 diff --git a/docs/decisions/ADR-0011-tui-testing-layers.md b/docs/decisions/ADR-0011-tui-testing-layers.md index f130ce7..cc62d51 100644 --- a/docs/decisions/ADR-0011-tui-testing-layers.md +++ b/docs/decisions/ADR-0011-tui-testing-layers.md @@ -14,3 +14,14 @@ The TUI is a state machine with a pure `render(state, frame)` and an `on_event` ## Consequences Both layers run headless under `cargo test`. The demo binary doubles as a visual smoke test. Snapshots are committed and reviewed like code. + +## Update — 2026-09-08 (termlens 0.10) + +The two layers stand; the dependency moved 0.9 → 0.10.1 and the PTY layer grew a third and fourth file (this ADR's own text above still said 0.6.1, which is what it was when the decision was taken — the manifest had since moved to 0.9 without the record following), because 0.10 made claims checkable that were previously only assumed: + +- `tests/emulation.rs` pins `Screen::unsupported()` — the sequences the emulator did *not* implement. Every other assertion in this crate reads a grid the emulator built, so a dropped sequence makes all of them plausible and wrong. Measured here: exactly `["^[[59m"]`, ratatui's underline-colour reset, which changes no cell. +- `tests/tui_pty_test.rs` keeps its three text snapshots byte-identical (`styles = false`; 0.10 records styles by default) and adds one styled snapshot plus direct cell assertions for the two things this dashboard says in colour alone: the focused panel's border and each operator's backend tag. `TestBackend::to_string()` cannot see either. +- `crates/oxidelake-runtime/tests/oxide_tui_pty.rs` extends the layer to the binary users install: `oxide tui` renders the same frame `oxidelake-tui` snapshots, and the `tracing` subscriber writing to stderr — the same stream as the dashboard in a PTY — puts nothing on the grid. +- `tests/termlens_cli.rs` reads the committed `.snap` files back with `termlens-cli`. It is `#[ignore]`d: these crates are published, and `cargo test` must not install a tool behind a contributor's back. CI runs it (`make test-termlens-cli`). + +The vendored agent skill is checked against the dependency by `make skill-version`, because it had already drifted two releases behind. diff --git a/docs/dependencies.md b/docs/dependencies.md index 915b583..d036993 100644 --- a/docs/dependencies.md +++ b/docs/dependencies.md @@ -29,9 +29,10 @@ The pins live in [SPEC.md §4](SPEC.md). This page records *why* those numbers, | `objc2-metal` | 0.3 | `objc2 >=0.6.2, <0.8`, `objc2-foundation ^0.3.2` | macOS target-gated, behind `metal` | | `objc2` / `objc2-foundation` | 0.6 / 0.3 | — | | | `io-uring` | 0.7 | — | Linux target-gated, behind `io-uring`; chosen over `tokio-uring` 0.5, which pins `io-uring ^0.6` and needs its own runtime ([ADR-0007](decisions/ADR-0007-io-uring-crate-dedicated-thread.md)) | -| `termlens` | 0.6 (dev) | default feature `insta` | real-PTY harness driving `oxidelake-tui-demo`; MSRV 1.85; read before use | +| `termlens` | 0.10 (dev) | default feature `insta`; feature `serde` | real-PTY harness driving `oxidelake-tui-demo` and `oxide tui`; MSRV 1.85; `serde` is what `TERMLENS_ARTIFACT_DIR` writes in CI and what `tests/emulation.rs` round-trips; read before use, and refresh `.claude/skills/termlens/SKILL.md` with the dependency (`make skill-version`) | | `insta` | 1 (dev) | — | snapshot files are committed | -| `assert_cmd` | 2 (dev) | — | CLI end-to-end tests, including the spawned scheduler + worker | +| `assert_cmd` | 2 (dev) | — | CLI end-to-end tests, including the spawned scheduler + worker; it captures pipes, so anything about a *terminal* is a termlens test | +| `serde_json` | 1 (dev) | — | reads a serialized termlens `Screen` back in `oxidelake-tui/tests/emulation.rs`; already in the lockfile through DataFusion | | `postcard` | 1 (`use-std`) | `serde` | codec payloads for `Gpu*Exec` parameters | | `rand` | 0.10 (dev) | — | seeded generators only | | `thiserror` / `anyhow` | 2 / 1 | — | libraries / binaries | diff --git a/docs/verification.md b/docs/verification.md index 68e2d22..6dd8c4a 100644 --- a/docs/verification.md +++ b/docs/verification.md @@ -8,7 +8,10 @@ CI separates default, io_uring and predict tests into three Linux jobs. Clippy also has default / CUDA / predict jobs. macOS runs Metal lint, Metal package tests together with the PTY suite, then on-device conformance when a device is available. The remaining checks are MSRV, all-feature rustdoc, `cargo deny`, -release scripts, crate metadata, CI policy tests and `zizmor`. `required-green` +release scripts, crate metadata, CI policy tests, the vendored termlens +skill's version and `zizmor`. The Linux and macOS test lanes run with +`TERMLENS_ARTIFACT_DIR` set and, on failure, render every screen the PTY +suite left behind into the job summary. `required-green` aggregates their results under the stable name required by branch protection. ```bash @@ -17,9 +20,27 @@ make test test-predict make test-io-uring # Linux only make check-cuda check-predict-no-second-cuda make doc coherence deny -make release-scripts crate-metadata ci-scripts zizmor +make release-scripts crate-metadata ci-scripts skill-version zizmor ``` +`make skill-version` compares the `Written against **termlens X.Y.Z**` line in +`.claude/skills/termlens/SKILL.md` with the `termlens` requirement in +`Cargo.toml` (major.minor). CI runs it as the `skill-version` job. + +One CI step is deliberately **not** in `make gate`: + +```bash +make test-termlens-cli # installs termlens-cli from crates.io +``` + +It runs the `#[ignore]`d `oxidelake-tui/tests/termlens_cli.rs` suite, which +drives the committed screens through the `termlens` command at the version +`Cargo.lock` names. The gate has to run on a machine with no network, and a +`cargo test` on a published crate must not install anything behind a +contributor's back — so CI asks for it by name, in the default Linux test +lane. The macOS on-device pass skips it (`--skip termlens_cli`) for the same +reason. + The Metal lane (macOS, since 2026-09-01 — CI runs it as the `metal` job on a macOS arm64 runner): @@ -128,7 +149,10 @@ cargo test -p oxidelake-compute --features cuda -- --ignored | SQL UDFs | `oxidelake-compute/src/udf.rs` | `l2_distance` / `cosine_distance` against the reference kernels, null and coercion cases | | DataFrame API | `oxidelake-api/tests/dataframe.rs` | every verb against its SQL equivalent; GPU-target plans show `Gpu*Exec` for fluent pipelines | | TUI in-process | `oxidelake-tui/tests/tui_render_test.rs` | `ratatui::backend::TestBackend` + `insta` | -| TUI end-to-end | `oxidelake-tui/tests/tui_pty_test.rs` | `termlens` driving `oxidelake-tui-demo` in a real PTY | +| TUI end-to-end | `oxidelake-tui/tests/tui_pty_test.rs` | `termlens` driving `oxidelake-tui-demo` in a real PTY: three text snapshots, one styled, and the focus/backend colours as cell assertions | +| TUI emulator invariants | `oxidelake-tui/tests/emulation.rs` | `Screen::unsupported()` pinned exactly, no terminal mode left set, snapshot-text and JSON round trips | +| TUI saved screens | `oxidelake-tui/tests/termlens_cli.rs` | `termlens-cli` on the committed `.snap` files — `render`, and `diff`'s 0/1/2 exit codes (`#[ignore]`d; `make test-termlens-cli`) | +| Shipped TUI binary | `oxidelake-runtime/tests/oxide_tui_pty.rs` | `oxide tui` in a PTY: the same frame `oxidelake-tui` snapshots, and no log line on the grid | | CLI end-to-end | `oxidelake-runtime/tests/cli.rs` | `assert_cmd` on `oxide` — `gen-data` output verified by an independent `parquet`-crate read; spawned `oxide-scheduler` + `oxide-worker` must print byte-identical `--cluster` results | ## Updating STATUS.md