Deterministic UI audit harness + map of what's broken

[17Amir17]

Summary

• Adds a repeatable, source-derived UI audit of every interactive affordance in apps/web — replacing ad-hoc "click around and see" passes that kept missing things — plus a checked-in map of what's broken (docs/audit/BROKEN-MAP.md).
• scripts/audit/: a TestSpec grammar (schema.md), a deterministic driver that boots an isolated vite+browser stack per shard (local in-browser engine) and drives each spec to PASS/FAIL/BLOCKED from objective evidence (page/console errors, feedback text, viewport pixel-diff, DOM state), a 5-phase multi-agent workflow (inventory → execute → adversarial verify → synthesize), and gen-map.mjs.
• docs/audit/: the enumerated test map (specs/*.json — 1,928 affordances across 25 areas), per-spec verdicts (results/*.json), curated evidence screenshots, and the map. Bulk screenshots are gitignored.

Results of the first full run

• 582 pass · 409 blocked · 937 raw failures. Blocked = untestable in this sandbox (no remote PyMOL bridge, no offscreen GL, no server file-service) — recorded as BLOCKED, never as passing.
• Adversarial re-verification rejected ~96% of raw failures as over-strict-check false positives and confirmed 11 real defects (4 distinct root causes):
  1. bg_color never repaints the live WebGL viewport (setting not wired to the renderer).
  2. Movie frame scrollbar permanently disabled (get_movie_status stub always returns nframes:0).
  3. Scene ▸ Blank throws "No scenes" instead of clearing the view (empty-key recall bug).
  4. Builder dock first-click race — quick button doesn't open the dock on first click; 7 controls unreachable.
• 77 low-signal candidates (mostly mouse/shortcuts, many "undrivable by design") were not re-verified before the sandbox hit its process limit; listed in the appendix, not counted as defects, with ready-to-run re-verify shards.

Test plan

• Read docs/audit/BROKEN-MAP.md — the funnel table and the 4 confirmed defects.
• Reproduce a defect: node scripts/audit/run-shard.mjs docs/audit/specs/viewport.json /tmp/out.json /tmp/shots and inspect viewport.render.bgColor.
• Spot-check a spec file (docs/audit/specs/menubar.json) against scripts/audit/schema.md.
• Optional: complete the unverified appendix after a session restart via the shards in docs/audit/reverify/.

🤖 Generated with Claude Code

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
10play	🟢 Ready	View Preview	Sep 3, 2026, 10:30 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

[claude]

blocking: hardcoded 7 doesn't match the data. confByArea.builder actually has 8 confirmed entries (see docs/audit/_summary.json — builder.bonds.order.triple, .model.rest, .model.sculpt, .bonds.delete, .settings.bumps, .atoms.clear.confirm, .wizard.cancel, .wizard.dismiss). This script is supposed to be deterministic/data-driven (per its own header comment and the README), but this count is a literal string, not computed from confByArea.builder.length. The checked-in docs/audit/BROKEN-MAP.md already shows the resulting contradiction: the headline and section-4 title both say "7", while the coverage table on the same page (line 51, computed from st.conf) correctly shows 8 for builder, and the "Affected controls" list right below the title enumerates 8 ids. Should be ${confByArea.builder?.length ?? 0} (or similar) in both this line and line 52's title, not a hardcoded number.

[claude]

blocking: same hardcoded-7 issue as line 43 — the group title says "7 controls unreachable" but ids (confByArea.builder) resolves to 8 entries at render time, so the title contradicts the "Affected controls" list generated a few lines below it for the same group.

[claude]

suggestion: The "7 of them are one Builder-dock bug" count is hardcoded rather than derived from confByArea.builder.length. The actual builder-area confirmed-defect count is 8 (see the table and the "Affected controls" list a few lines below, both of which correctly show 8), so the generated docs/audit/BROKEN-MAP.md headline currently contradicts its own table/list. Suggest computing this from confByArea.builder?.length ?? 0 so it can't drift again on the next regen.

[claude]

[blocking] Docs contradict the code they describe.

This PR removes the apbs dialog leaf (and the whole features/apbs/** implementation, per scripts/migrate/plan/remove-impossible.json), but docs/feature-parity.md:481-483 still asserts, unedited:

• Row "APBS Electrostatics — menu entry stays visible" ([x], i.e. confirmed done) — the menu entry does not stay visible anymore; it was deleted here.
• "Covered by" cites apps/web/src/features/apbs/p13apbsEntry.dom.test.tsx, ApbsPanel.dom.test.tsx, apbsProbe.test.ts — all three were deleted in this same PR (commit 735449a).

docs/code-ownership.md:247 (WP-30) also still lists apps/web/src/features/apbs/** as an owned path that no longer exists.

docs/feature-parity.md describes itself as "a reference, not a work log" and its [x]/[-] verdict column exists precisely to record descoping decisions like this one — the row should be updated to [-] (descoped, browser-only decision) rather than left claiming the feature is present with tests that no longer exist. The removal's own plan (scripts/migrate/plan/remove-impossible.json) has no task for updating these docs, which is the gap.

[claude]

blocking: dpi is passed both positionally and as a kwarg here.

args: ['', 0, 0, -1] fills PyMOL's cmd.png(filename, width, height, dpi=…) signature through the dpi slot (index 3, value -1), and then { prior: 1, dpi } passes dpi again as a keyword. Over the real PyMOL bridge this reaches target(*call_args, **call_kwargs) (packages/bridge/tenmol_bridge/dispatch.py:143) unchanged, i.e. cmd.png('', 0, 0, -1, prior=1, dpi=<value>) — Python raises TypeError: png() got multiple values for argument 'dpi'. The local engine-ts backend masks this because its pick() helper (packages/engine-ts/src/cmd/render.ts:109) tolerates the duplicate, and wfExportPng.dom.test.tsx never exercises this RenderPanel Draw/Ray save path, so nothing catches it.

The existing bridge convention avoids exactly this (see panels/files.py::copy_image_png, which only ever passes filename positionally and puts everything else in kwargs). This call should do the same, e.g. session.call('cmd.png', [''], { prior: 1, dpi }).

[claude]

blocking: fileOpen no longer routes dialog-needed formats (session/map/mtz/mae/trajectory/alignment) to their modal — it silently text-decodes and cmd.loads everything except .pwg.

Before this PR, uploadAndLoad/fileOpen went through load() (FileDropTarget.tsx), which calls dialogNeededFor(classify(...)) and shows dialogRequiredMessage for traj/map/mtz/aln/mae/session before ever touching cmd.load. This new fileOpen (and the equivalent uploadAndLoad in FileDropTarget.tsx:156-171, used by drag-drop and Ctrl+O) only checks refusalFor(browserClassification(...)), which exclusively recognizes .pwg. Every other format — including binary ones like .pse/.mtz/.ccp4/.dsn6 — falls through to file.text() (UTF-8-decoding binary bytes, corrupting them) and an unconditional cmd.load(content, name, 0, '').

The inline comment on FileDropTarget.tsx:152-154 claims "Dialog-only formats … keep their planFromDataTransfer/dialogNeededFor handling on the URL branch below" — but that's only true for the plan.kind === 'url' branch; the files branch (the common case: local Open…/drag-drop/Ctrl+O) never calls dialogNeededFor at all. wfBrowserOpen.dom.test.tsx doesn't test any of these formats, so this regression isn't caught by the new suite.

Opening a .pse (partial-session merge), .mtz/map (import settings), .mae (Maestro), or alignment file via File ▸ Open…, Ctrl+O, or drag-and-drop will now silently mis-load or fail instead of prompting the user with the required dialog.

[claude]

blocking: Dropping/opening a .pml, .py, or .pym script now silently fails.

classify() maps these extensions to dialog: 'script' (globalDrop.ts:164-166), and dialogNeededFor deliberately excludes 'script' from NEEDS_DIALOG — the comment there says "a .pml/.py drop is a cd-then-run, which needs no user input", so it falls through to this content-based cmd.load(content, objectName, 0, '') call with a blank format.

But engine-ts's load command (packages/engine-ts/src/cmd/fileio.ts) has no script/pml branch at all — script execution only happens through the separate run/@include path (controlflow.ts, parser.test.ts). For multi-line content with no recognizable structure marker, format = fmtArg || sniffFormat(content) || extFormat resolves to '', and load throws load: could not determine the structure format of the given content (fileio.ts ~line 999).

So every browser-native Open…/drop of a script file now produces a confusing "unknown format" engine error instead of running the script (or a clear "scripts aren't supported in the browser build" message, consistent with how refused/dialog-needed formats are handled elsewhere in this same PR). Same issue at FilesPanel.tsx:438.

[claude]

suggestion: browserClassification parses the extension from the raw name without stripping a .gz/.bz2 wrapper first, unlike classify() a few lines above which does (base.replace(/\.(gz|bz2)$/i, '')). Currently harmless since the only refused extension is pwg, but if CLIENT_REFUSED_FORMATS ever grows, a foo.pwg.gz would silently skip the refusal that foo.pwg gets. Worth aligning the two for consistency.

[claude]

suggestion: exportMolecule now refuses gracefully when request.multisave || request.pattern is set (browser can't produce multiple files from one download), but SaveDialogs.tsx still renders the multisave checkbox and the "Multiple files" pattern radios as if they work. Consider disabling/hiding those controls in the browser build so the dialog doesn't offer options that always end in a refusal message.

[claude]

blocking: sessionSaveAs downloads whatever cmd.get_session resolves to without checking for a blob handle. On the remote/PyMOL bridge backend, get_session is in the bridge's BLOB_RETURNS set (packages/bridge/tenmol_bridge/codec.py:93-107), so session.call('cmd.get_session') resolves to {__blob__: true, id, mime, size, url, ...} (packages/bridge/tenmol_bridge/blobs.py:256-271), not a session object — connection.ts returns message.result verbatim with no blob auto-fetch (packages/client/src/connection.ts:634). JSON.stringify(snap) then downloads a tiny JSON stub referencing a server temp blob (which is later evicted) instead of the actual .pse bytes real PyMOL wrote. The comment on line 462 ("works over the remote bridge too") is incorrect for this reason.

This repo already has the fix pattern: apps/web/src/features/volume/service.ts:44-56 defines BlobHandle/isBlob and fetches the real bytes via fetch(session.config.httpOrigin + value.url) when a call resolves to a blob handle. sessionSaveAs (and the duplicated Ctrl+S handler in FileDropTarget.tsx:259-260) should do the same before serializing/downloading.

No existing test catches this — p9f1MenuHooks.dom.test.tsx's REPLIES['cmd.get_session'] stub is always a plain object, never a blob handle, so the remote-backend path is untested.

[claude]

blocking: same blob-handle bug as FilesPanel.tsx's sessionSaveAs (see that inline comment) — this is the duplicated Ctrl+S implementation. cmd.get_session is a BLOB_RETURNS symbol on the remote bridge, so snap here can be {__blob__:true, id, url, ...} rather than a session object; JSON.stringify(snap) downloads that stub instead of the real .pse bytes. Needs the same isBlob/fetch(session.config.httpOrigin + value.url) handling used in apps/web/src/features/volume/service.ts:44-56, 265-266.

[claude]

suggestion: this Ctrl+S handler duplicates FilesPanel.tsx's sessionSaveAs almost line-for-line (prompt, extension defaulting, get_session call, downloadText). Once the blob-handle bug above is fixed, it'd be worth extracting a single shared helper (e.g. in download.ts or a new sessionSave.ts) so the two call sites can't drift again.