SCAL-320648 mcp changes to support spotter model through tools

[pradeepburugula]

Summary

Adds a new V3 tool surface ("Spotter Model") that lets an MCP client build a ThoughtSpot data model conversationally through the Lumos agentic backend. The agent plans, then auto-builds tables/joins/columns, and the model is saved as a worksheet — all driven by four new tools.

Branch spotter-model-mcp (on fork pradeepburugula/mcp-server), rebased onto the latest main (#195 cluster blocklist); one commit ahead.

New tools

Tool	Purpose
create_model_session	Open a Lumos model-building session on a warehouse connection. Returns a model_session_id.
send_model_message	Send a natural-language instruction/answer; streams the builder's response for the turn.
get_model_updates	Continuation long-poll for turns that outlive a single send window (e.g. the ~1–2 min auto-build).
finalize_model	Review, then (on confirm) persist the model and return its URL.

Flow: create_model_session → send_model_message → get_model_updates (as needed) → finalize_model.

Implementation notes

• Streaming: send_model_message opens the Lumos SSE stream (surfacing auth/conn errors synchronously), consumes it in the background into a normalized ModelUpdate stream, and long-polls storage inline so short turns complete in one call.
• Durable storage: session state (model-put/model-get) + an append-only, bookmarked update stream (model-append/model-updates) in the existing ConversationStorageServerSQLite DO — additive tables/routes, no new DO class or migration.
• Correctness: choice-binding by transaction_id+generation_no (Lumos sends no choice_id); generation threading coerces string→number and accumulates the working set; save requires the full working set.
• MRD (plan-then-approve): planning turns are surfaced as a distinct mrd update so a client won't finalize an unbuilt plan.
• Save: default path is the bach worksheet-editor op (/prism/?op=WorksheetOperation), matching the modelling UI's Save. A SAVE_VIA_LUMOS env flag (default off) routes save through Lumos instead — unverified end-to-end, gated intentionally.
• Polling: get_model_updates/send use a ~60s server-side long-poll window, chosen to keep round-trips low without hitting the MCP connector's per-call timeout.
• Versioning: registered as latest (2026-07-31); the prior V2 date is now a pinned entry.

Config

• SAVE_VIA_LUMOS (default "false") — feature flag for the Lumos save path.

Reviewer guide (suggested reading order)

#	File	What it is / what to look at
1	src/servers/tool-definitions.ts	V3 API surface — Zod schemas + tool descriptions + the ModelUpdate shape. Read first for the contract.
2	src/servers/mcp-server.ts	Core handlers & logic (bulk of the diff): the 4 call* handlers, consumeModelStreamToStorage (SSE→updates), pollModelUpdates, buildChoiceAnswer, advanceGeneration, looksLikeMrd, isSaveViaLumosEnabled.
3	src/thoughtspot/thoughtspot-client.ts	External calls — Lumos /chat/init (ChatInitV2), /chat/{id}/stream, mintSessionCookie; save via bach /prism (default) and saveModelViaLumos (flagged).
4	src/thoughtspot/thoughtspot-service.ts	Thin service wrappers — low-scrutiny.
5	src/thoughtspot/types.ts	ModelSessionState, ModelUpdate, ModelUpdatesState.
6	src/servers/conversation-storage-server.ts	DO storage: scalar session state + append-only bookmarked update stream (read/write bookmarks + done flag). Additive routes on the existing DO.
7	src/storage-service/storage-service.ts	StorageServiceClient methods calling the DO routes.
8	src/servers/version-registry.ts	V3 registered as latest (2026-07-31).
9	wrangler.jsonc	Only the SAVE_VIA_LUMOS var added.
10	test/**	Updated tool-count + version-date expectations.

Where to spend attention: streaming lifecycle + the ~60s window vs. connector timeout; choice binding & generation threading; the looksLikeMrd heuristic; the default-vs-Lumos save path; DO stream concurrency.

Testing

• Full flow verified live on a dev cluster: create → plan (MRD) → auto-build → get_model_updates → generation 2→3→4, producing a 7-table / 6-join / 50-column model → finalize review.
• Unit tests updated; full suite green (648 passing) after rebasing onto latest main. No new tsc/biome errors beyond the repo baseline.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.