Skip to content

HTLM as grounder behind a strong planner — controlled A/B on browser-use BU Bench #7

Description

@espetro

Test the real thesis: does a strong LLM (planner) + HTLM (grounder) beat the strong LLM doing its own grounding, on a live multi-step browser benchmark?

Controlled A/B, host tool browser-use (Python), benchmark browser-use BU Bench V1, planner via OpenRouter. Only the grounder call differs between arms:

  • Baseline: strong LLM grounds its own sub-instruction.
  • Treatment: HTLM local server grounds it.

Plan: .agents/plans/2026-07-15-htlm-planner-grounder-benchmark.md

Deliverables:

  • integration/htlm_server.py — local HTTP grounder (llama-cpp-python, GGUF, reuses data/pipeline/prompt.py + training/eval.py::_extract_action)
  • integration/adapter.py — browser-use DOM ⇄ HTLM page-representation/action schema bridge
  • integration/planner.py — OpenRouter strong-LLM planner + shortlist pruning
  • integration/loop.py — shared step loop, grounder strategy param (llm|htlm)
  • integration/run_bench.py — BU Bench subset (n=20) runner + comparison table

Success = comparison table shows delta (HTLM arm vs LLM-grounding arm) on success rate, latency, cost. Ship integration only if delta favorable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureClassification: feature

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions