Skip to content

fix(deploy-benchmark): add --subset flag for tau2 domain selection - #53

Open
vz-ibm wants to merge 1 commit into
rossoctl:mainfrom
vz-ibm:fix/deploy-benchmark-subset-flag
Open

fix(deploy-benchmark): add --subset flag for tau2 domain selection#53
vz-ibm wants to merge 1 commit into
rossoctl:mainfrom
vz-ibm:fix/deploy-benchmark-subset-flag

Conversation

@vz-ibm

@vz-ibm vz-ibm commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Problem

lib/sparc-run.sh and lib/baseline-run.sh (in the downstream rosso-sparc
integration scripts) call deploy-benchmark.sh ... --subset "$SUBSET" to
select a tau2 domain (retail/airline/mock/telecom), but deploy-benchmark.sh
has never actually supported that flag:

Error: Unknown option: --subset

Every call silently failed the entire deploy, or was worked around by never
passing --subset at all (defaulting to whatever domain the image was baked
with — normally retail), which made running the airline domain (or any
non-default domain) impossible via this script.

Fix

Adds --subset NAME as a proper CLI flag. It's wired as a bare SUBSET env
var on the deployed container (not through the existing EXGENTIC_SET_*
--set-style mechanism), matching how exgentic-mcp-tau2's own
entrypoint.sh reads it directly (--subset $SUBSET) — that support landed
in Exgentic/exgentic feature/mcp-command commit ae4f1c9.

Requires an exgentic-mcp-tau2 image built from that commit or later; older
images silently ignore the env var and just run their build-time default
domain.

Verification

Ran end-to-end on a live cluster: rebuilt the tau2 MCP image from current
feature/mcp-command, deployed with --subset airline, confirmed via pod
logs:

Command: exgentic mcp --benchmark tau2 --host 0.0.0.0 --port 8000 --subset airline ...
✓ Loaded benchmark 'tau2' with 50 tasks

then ran the full 50-task airline baseline through to completion:
Sessions Attempted: 50, Sessions Succeeded: 50, Sessions With Error: 0.

lib/baseline-run.sh and lib/sparc-run.sh already called
deploy-benchmark.sh --subset "$SUBSET" for airline/retail selection, but
the flag didn't exist -- it was a silent no-op error ("Unknown option:
--subset"). Wires it as a bare SUBSET env var (not the EXGENTIC_SET_*
--set mechanism) matching entrypoint.sh's `--subset $SUBSET` read,
added upstream in Exgentic/exgentic feature/mcp-command commit ae4f1c9.
Requires an exgentic-mcp-tau2 image built from that commit or later.
@vz-ibm
vz-ibm force-pushed the fix/deploy-benchmark-subset-flag branch from 6da4fd3 to 00ee312 Compare August 5, 2026 08:16
@vz-ibm

vz-ibm commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

These PR blocked by xGentic PR#187

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant