Skip to content

In-browser live grounding demo (proof-of-work) #8

Description

@espetro

Self-contained in-browser demo: 350M GGUF loaded client-side via wllama (no server in the inference path), user types an instruction against a real page snapshot, model streams {type,index,value}, chosen element highlighted. The "runs in YOUR browser" proof, watchable and reproducible — sits next to the offline 91.2% strict action accuracy (Mind2Web held-out) as live supporting evidence.

Plan: .agents/plans/2026-07-16-live-proof-of-work.md (Track B)

Deliverables:

  • runtime-bench/demo.html — interactive page (sibling to harness.html), loads GGUF via wllama, uses wllama.createChatCompletion so llama.cpp applies the GGUF's embedded chat template, renders returned action + highlights chosen element/bounds.
  • Verification prerequisite: confirm exported GGUF carries tokenizer.chat_template metadata and its templated output matches training/eval.py::_build_prompt byte-for-byte for a known record; fall back to inlining the LFM2.5 template in JS if not.
  • Curated seed snapshots/instructions (reuse data/samples/*.jsonl) so the demo works on open with zero setup.
  • README.md — "Try it live (in your browser)" section with recorded GIF + one-command serve.py invocation.

Success = a seeded task grounds to the same action the offline eval expects, and a fresh instruction against a pasted snapshot highlights the correct element, recorded as a GIF for the README.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureClassification: feature

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions