No-assets mode: the full benchmark with zero RCT2 files, Linux images, OpenAI-compatible lane - #2

Open

tlrmchlsmth wants to merge 18 commits into

wseaton:evalfrom

tlrmchlsmth:linux-support

Commits on Jul 24, 2026

No-assets mode: run the whole eval without RCT2 files (--no-graphics)

tlrmchlsmth
and
claude
committed
ac3978b
CI image: build the orphaned graphics target; verified on Linux arm64

tlrmchlsmth
and
claude
committed
c0c1667
OpenAI lane: retry when tool_choice=required returns no tool call

tlrmchlsmth
and
claude
committed
c5666e8
driver: --max-tokens, fail fast when thinking exhausts the budget

tlrmchlsmth
and
claude
committed
6da1d26
CI notes: document the vLLM findings from the first live runs

tlrmchlsmth
and
claude
committed
cb35fe7
driver: named tool_choice is advisory on some stacks; retry forced submit

tlrmchlsmth
and
claude
committed
b122f5e
driver: COASTERBENCH_CLI env override for CI binary location

tlrmchlsmth
and
claude
committed
b952e70
CI notes: final attribution — one confirmed vLLM bug, one anomaly, one harness fix

tlrmchlsmth
and
claude
committed
6d66daa
driver: force submission in-band when the validation budget runs out

tlrmchlsmth
and
claude
committed
81da8a0
Asset-free track graphics: cursor trace in the report, schematic PNG per round

tlrmchlsmth
and
claude
committed
5911936
CI image: bundle the smoke program alongside the scenario

tlrmchlsmth
and
claude
committed
6dbbff5
OpenAI lane: preserve reasoning across turns

tlrmchlsmth
and
claude
committed
421cf25
driver: default to 6 rounds

tlrmchlsmth
and
claude
committed
f50817c
CI image: fix objects prefix bug; add self-contained eval stage

tlrmchlsmth
and
claude
committed
6016489
driver: COASTERBENCH_OPENRCT2_DATA env for containerized data path

tlrmchlsmth
and
claude
committed
4e7630d

Commits on Jul 25, 2026

OpenAI lane: hour-long request timeout for thinking models

tlrmchlsmth
and
claude
committed
4abde85
CI notes: thinking-tier verdict — unbounded reasoning on one-shot prompts

tlrmchlsmth
and
claude
committed
c1c8d47
driver: save the raw response when a call exhausts its budget

tlrmchlsmth
and
claude
committed
ea30482

Navigation Menu

No-assets mode: the full benchmark with zero RCT2 files, Linux images, OpenAI-compatible lane - #2

No-assets mode: the full benchmark with zero RCT2 files, Linux images, OpenAI-compatible lane#2
tlrmchlsmth wants to merge 18 commits into
wseaton:evalfrom
tlrmchlsmth:linux-support

Commits on Jul 24, 2026

No-assets mode: run the whole eval without RCT2 files (--no-graphics)

CI image: build the orphaned graphics target; verified on Linux arm64

OpenAI lane: retry when tool_choice=required returns no tool call

driver: --max-tokens, fail fast when thinking exhausts the budget

CI notes: document the vLLM findings from the first live runs

driver: named tool_choice is advisory on some stacks; retry forced submit

driver: COASTERBENCH_CLI env override for CI binary location

CI notes: final attribution — one confirmed vLLM bug, one anomaly, one harness fix

driver: force submission in-band when the validation budget runs out

Asset-free track graphics: cursor trace in the report, schematic PNG per round

CI image: bundle the smoke program alongside the scenario

OpenAI lane: preserve reasoning across turns

driver: default to 6 rounds

CI image: fix objects prefix bug; add self-contained eval stage

driver: COASTERBENCH_OPENRCT2_DATA env for containerized data path

Commits on Jul 25, 2026

OpenAI lane: hour-long request timeout for thinking models

CI notes: thinking-tier verdict — unbounded reasoning on one-shot prompts

driver: save the raw response when a call exhausts its budget