lidge-jun · Wibias · Aug 12, 2026 · Aug 11, 2026 · Aug 11, 2026 · Aug 11, 2026
@@ -0,0 +1,62 @@
+# Web-search sidecar: opt-in live streaming (`streamRoutedModelOutput`)
+
+Date: 2026-08-12. Follow-up to `260806_codex_desktop_streaming/000_findings.md` — this identifies
+the concrete cause of the "chat answers arrive as one end-of-turn burst" symptom that investigation
+left open, and lands the fix.
+
+## Root cause (reproduced end-to-end)
+
+Codex CLI/Desktop sends a hosted `web_search` tool on every real turn. For routed (non-passthrough)
+models with a usable ChatGPT credential, `planWebSearch` engages the web-search sidecar, and
+`runWithWebSearch` → `consumeIterationEvents` **fully buffers** every semantic adapter event of an
+iteration before scanning for `web_search` calls. Client-visible output therefore arrives only at
+turn end — 6–50 s of silence on reasoning-heavy turns, then a burst.
+
+Evidence chain (all on one machine, same provider `opencode-go/deepseek-v4-flash`, same key):
+
+- Two bit-identical proxy installs behaved differently: the instance with ChatGPT auth buffered
+  (`firstOutputMs ≈ durationMs` on 99/103 conversation requests), the instance with an EMPTY
+  `CODEX_HOME` streamed (`firstOutputMs ≈ 1.5–3 s`) — because only the former could engage the
+  sidecar.
+- Byte-identical replay of a captured real `codex exec` request: buffered on the sidecar-enabled
+  instance, streamed on the other. Field bisect: removing only the `web_search` tool made the
+  sidecar-enabled instance stream (first delta 3.5 s, 448 deltas); removing `tool_search` /
+  `namespace` tools did not.
+
+## Why buffering exists, and what the fix preserves
+
+Buffering keeps two invariants: (1) the synthetic `web_search` tool call must never leak to Codex,
+and (2) preliminary output from a pre-search iteration must not surface as the answer. The fix
+keeps both by construction:
+
+- Live delivery is **opt-in** (`webSearchSidecar.streamRoutedModelOutput`, default `false`).
+- Only event types the sidecar-less path would deliver identically may leave the live window:
+  `text_delta`, `thinking_delta`, `reasoning_raw_delta`, `thinking_signature`,
+  `redacted_thinking`, `kiro_redacted_reasoning` (allowlist in `loop.ts`).
+- The window closes permanently at the first buffer-only event — tool calls above all — so the
+  `web_search` interception decision stays atomic and live events are exactly the first N
+  passthrough entries. The terminal replay skips them by count; nothing is delivered twice.
+- Scanner semantics are unchanged: live events are still buffered for `extractIterationThinking`
+  and the forced-answer output check (#1001 behavior intact).
+
+Accepted tradeoff (documented in `docs-site/.../sidecars.md`): text the model emits before deciding
+to search — which buffered mode silently drops — becomes visible and may partially repeat in the
+post-search answer. Reasoning-first models (the common case) avoid the text-repetition case, though
+their leading reasoning deltas become client-visible too — that visibility is the point of the
+option.
+
+## Verification
+
+- `bun test tests/web-search.test.ts` — 55 pass, including 4 new tests: a gated adapter proves
+  deltas reach the client while the adapter is still mid-turn (buffered mode would deadlock the
+  gate); default-off buffering; window close at `tool_call_start` with exactly-once replay of the
+  tail; search-loop pass with pre-search text delivered exactly once.
+- `bun test tests/web-search-*.test.ts` — 78 pass. `bun x tsc --noEmit` clean.
+- Live replay of the captured Codex request through a patched instance (sidecar-less path):
+  893 deltas, first at 2.6–3.3 s, unchanged totals.
+- Sidecar-ACTIVE live E2E (patched build running as the native-main owner with a real ChatGPT
+  credential, identical text-forcing request, routed `opencode-go/deepseek-v4-flash`): toggle off →
+  18.3 s silence then 2829 deltas in one 0.02 s burst; toggle on → first delta at 4.0 s, 2538
+  deltas over 9.9 s. The toggle applied without restart via `PUT /api/sidecar-settings`.
+  Field note: a paused ChatGPT account (`pausedCodexAccountIds`) silently disables the sidecar and
+  masks both bug and fix — everything streams because the sidecar-less path runs.
@@ -29,10 +29,25 @@ When Codex requests hosted `web_search` for a non-passthrough routed model, open
    (default 3), then removes the search tool and forces a final answer. Real client tools such as
    `apply_patch` or shell finalize the turn so those calls reach Codex.
 
-Every routed-model iteration requests upstream `stream: true`, but opencodex fully buffers semantic
-events internally before deciding whether to search or return the final answer. Only the first
-iteration's final headers/status and 429 key rotations are acquired eagerly. Thus synthetic search
-calls and preliminary output are never exposed as client-visible model output.
+Every routed-model iteration requests upstream `stream: true`, but by default opencodex fully
+buffers semantic events internally before deciding whether to search or return the final answer.
+Only the first iteration's final headers/status and 429 key rotations are acquired eagerly. Thus
+synthetic search calls and preliminary output are never exposed as client-visible model output.
+
+Opt-in `webSearchSidecar.streamRoutedModelOutput` (default `false`) streams each iteration's
+leading text/thinking deltas live instead — the client sees output as soon as the model produces
+it, exactly like the sidecar-less path. The live window closes permanently at the first tool-call
+boundary, so the decision to intercept `web_search` stays atomic and nothing is ever delivered
+twice (the terminal replay skips what already streamed). Tradeoff: text the model emits *before*
+deciding to search — which buffered mode silently drops — becomes visible and may partially repeat
+in the post-search answer. The Dashboard overview page exposes this as the **Stream answers live**
+toggle on the web-search sidecar card (`PUT /api/sidecar-settings` with
+`webSearch.streamRoutedModelOutput`).
+
+Kiro commentary is independent of this option: commentary-phase text already streams ahead of the
+terminal event in buffered mode, and that bypass is unchanged — with or without
+`streamRoutedModelOutput`, only search-decision events (tool calls and everything after the first
+tool-call boundary) remain buffered for the atomic `web_search` decision.
 
 The injected result is wrapped in an untrusted-data boundary, length-capped, and de-duplicated by
 source URL. In structured-output turns (`json_schema` / `json_object`) it is handed over as compact
@@ -48,7 +63,8 @@ relevant images in words and include their source URLs.
     "reasoning": "low",
     "maxSearchesPerTurn": 3,
     "routedModelStallTimeoutMs": 200000,
-    "timeoutMs": 200000
+    "timeoutMs": 200000,
+    "streamRoutedModelOutput": false
   }
 }
 ```

@@ -269,6 +269,8 @@ export const de: Record<TKey, string> = {
   "dash.visionModelHint": "Modell zur Beschreibung von Bildern für nur-Text-Routen. Erfordert ChatGPT-Login.",
   "dash.webSearchSidecar": "Websuche-Sidecar",
   "dash.webSearchSidecarHint": "Backend und Modell für die Websuche gerouteter Modelle auswählen.",
+  "dash.webSearchStream": "Antworten live streamen",
+  "dash.webSearchStreamHint": "Führenden Text und Reasoning live streamen, bis das Modell über einen Tool-Aufruf entscheidet; der Rest bleibt für das Abfangen der Suche gepuffert. Text vor einer Suche kann sich teilweise wiederholen.",
   "dash.visionSidecar": "Vision-Sidecar",
   "dash.visionSidecarHint": "Backend und Modell zur Bildbeschreibung für reine Textmodelle auswählen.",
   "dash.shadowCallIntercept": "Shadow-Call-Abfangen",

@@ -281,6 +281,8 @@ export const en = {
   "dash.visionModelHint": "Model used to describe images for text-only routed models. Requires ChatGPT login.",
   "dash.webSearchSidecar": "Web search sidecar",
   "dash.webSearchSidecarHint": "Choose the backend and model used for web search on routed models.",
+  "dash.webSearchStream": "Stream answers live",
+  "dash.webSearchStreamHint": "Stream the model’s leading text and reasoning live until it decides on a tool call; the rest of the turn stays buffered for search interception. Text written before a search may partially repeat.",
   "dash.visionSidecar": "Vision sidecar",
   "dash.visionSidecarHint": "Choose the backend and model used to describe images for text-only routed models.",
   "dash.shadowCallIntercept": "Shadow Call Intercept",

@@ -278,6 +278,8 @@ export const ja: Record<TKey, string> = {
   "dash.visionModelHint": "テキスト専用ルーティングモデルで画像を説明するために使うモデル。ChatGPT ログインが必要です。",
   "dash.webSearchSidecar": "ウェブ検索サイドカー",
   "dash.webSearchSidecarHint": "ルーティングモデルでウェブ検索に使うバックエンドとモデルを選択します。",
+  "dash.webSearchStream": "回答をライブ配信",
+  "dash.webSearchStreamHint": "モデルがツール呼び出しを決定するまで、先頭のテキストと推論をライブ配信します。以降は検索インターセプトのためバッファされます。検索前のテキストは一部繰り返される場合があります。",
   "dash.visionSidecar": "ビジョンサイドカー",
   "dash.visionSidecarHint": "テキスト専用ルーティングモデルで画像を説明するために使うバックエンドとモデルを選択します。",
   "dash.shadowCallIntercept": "シャドウコール傍受",

@@ -273,6 +273,8 @@ export const ko: Record<TKey, string> = {
   "dash.visionModelHint": "텍스트 전용 라우팅 모델에 이미지를 설명하는 데 사용되는 모델입니다. ChatGPT 로그인 필요.",
   "dash.webSearchSidecar": "웹 검색 사이드카",
   "dash.webSearchSidecarHint": "라우팅 모델의 웹 검색에 쓸 백엔드와 모델을 고릅니다.",
+  "dash.webSearchStream": "응답 실시간 스트리밍",
+  "dash.webSearchStreamHint": "모델이 도구 호출을 결정할 때까지 앞부분 텍스트와 추론을 실시간 스트리밍합니다. 이후는 검색 가로채기를 위해 버퍼링됩니다. 검색 전 텍스트가 일부 반복될 수 있습니다.",
   "dash.visionSidecar": "비전 사이드카",
   "dash.visionSidecarHint": "텍스트 전용 라우팅 모델이 이미지를 읽을 때 쓸 백엔드와 모델을 고릅니다.",
   "dash.shadowCallIntercept": "쉐도우 호출 가로채기",

@@ -278,6 +278,8 @@ export const ru: Record<TKey, string> = {
   "dash.visionModelHint": "Модель, которая описывает изображения для маршрутизируемых моделей, работающих только с текстом. Требуется вход в аккаунт ChatGPT.",
   "dash.webSearchSidecar": "Сайдкар веб-поиска",
   "dash.webSearchSidecarHint": "Выберите бэкенд и модель, используемые для веб-поиска на маршрутизируемых моделях.",
+  "dash.webSearchStream": "Стримить ответы вживую",
+  "dash.webSearchStreamHint": "Транслировать начальный текст и рассуждения вживую, пока модель не решит вызвать инструмент; остальное буферизуется для перехвата поиска. Текст до поиска может частично повторяться.",
   "dash.visionSidecar": "Сайдкар для изображений",
   "dash.visionSidecarHint": "Выберите бэкенд и модель, которые описывают изображения для маршрутизируемых моделей, работающих только с текстом.",
   "dash.shadowCallIntercept": "Перехват теневых вызовов",

@@ -279,6 +279,8 @@ export const tr: Record<TKey, string> = {
   "dash.visionModelHint": "Salt metin yönlendirilen modeller için görselleri tanımlamakta kullanılan model. ChatGPT girişi gerektirir.",
   "dash.webSearchSidecar": "Web arama yan aracı (sidecar)",
   "dash.webSearchSidecarHint": "Yönlendirilen modellerde web araması için kullanılan arka ucu ve modeli seçin.",
+  "dash.webSearchStream": "Yanıtları canlı akıt",
+  "dash.webSearchStreamHint": "Model bir araç çağrısına karar verene kadar baştaki metni ve akıl yürütmeyi canlı akıtır; kalanı arama yakalama için arabelleğe alınır. Aramadan önce yazılan metin kısmen tekrarlanabilir.",
   "dash.visionSidecar": "Görsel yan aracı (sidecar)",
   "dash.visionSidecarHint": "Salt metin modeller için görselleri tanımlamakta kullanılan arka ucu ve modeli seçin.",
   "dash.shadowCallIntercept": "Gölge Çağrı Yakalama",

@@ -172,6 +172,8 @@ export const zhTW: Record<TKey, string> = {
   "dash.visionModelHint": "為純文字路由模型描述圖像的模型。需要 ChatGPT 登入。",
   "dash.webSearchSidecar": "網頁搜尋附屬服務",
   "dash.webSearchSidecarHint": "選擇路由模型進行網頁搜尋時使用的後端和模型。",
+  "dash.webSearchStream": "即時串流輸出回答",
+  "dash.webSearchStreamHint": "即時串流輸出開頭的文字和推理，直到模型決定呼叫工具；其餘部分為攔截搜尋而保持緩衝。搜尋前的文字可能會部分重複。",
   "dash.visionSidecar": "視覺附屬服務",
   "dash.visionSidecarHint": "選擇純文字路由模型描述圖像時使用的後端和模型。",
   "dash.shadowCallIntercept": "影子呼叫攔截",

@@ -273,6 +273,8 @@ export const zh: Record<TKey, string> = {
   "dash.visionModelHint": "为纯文本路由模型描述图像的模型。需要 ChatGPT 登录。",
   "dash.webSearchSidecar": "网页搜索附属服务",
   "dash.webSearchSidecarHint": "选择路由模型进行网页搜索时使用的后端和模型。",
+  "dash.webSearchStream": "实时流式输出回答",
+  "dash.webSearchStreamHint": "实时流式输出开头的文本和推理，直到模型决定调用工具；其余部分为拦截搜索而保持缓冲。搜索前的文本可能会部分重复。",
   "dash.visionSidecar": "视觉附属服务",
   "dash.visionSidecarHint": "选择纯文本路由模型描述图像时使用的后端和模型。",
   "dash.shadowCallIntercept": "影子调用拦截",

@@ -284,6 +284,21 @@ export function DashboardSidecarPanels({ d }: { d: Dash }) {
               disabled={!sidecar || sidecarSaving}
               label={t("dash.sidecarModel")}
             />
+            <div style={{ display: "flex", alignItems: "center", gap: 8 }} title={t("dash.webSearchStreamHint")}>
+              <span className="muted setting-hint">{t("dash.webSearchStream")}</span>
+              <button
+                type="button"
+                className={`switch ${sidecar?.webSearch.streamRoutedModelOutput ? "on" : ""}`}
+                onClick={() => {
+                  void saveSidecar({ webSearch: { streamRoutedModelOutput: !sidecar?.webSearch.streamRoutedModelOutput } });
+                }}
+                disabled={!sidecar || sidecarSaving}
+                aria-label={t("dash.webSearchStream")}
+                aria-pressed={sidecar?.webSearch.streamRoutedModelOutput === true}
+              >
+                <span className="knob" />
+              </button>
+            </div>
           </div>
         </div>
 

@@ -56,7 +56,7 @@ export interface SettingsData {
 }
 export type SidecarBackend = "openai" | "anthropic";
 export type VisionReasoning = "low" | "medium" | "high" | "xhigh" | "max";
-export interface SidecarSetting { backend?: SidecarBackend; model: string; reasoning?: VisionReasoning }
+export interface SidecarSetting { backend?: SidecarBackend; model: string; reasoning?: VisionReasoning; streamRoutedModelOutput?: boolean }
 export interface VisionModelOption { value: string; label: string; backend: SidecarBackend; baseline?: boolean }
 export interface SidecarData {
   webSearch: SidecarSetting;
@@ -67,7 +67,7 @@ export interface SidecarData {
   visionModels?: VisionModelOption[];
 }
 export interface SidecarPatch {
-  webSearch?: { backend?: SidecarBackend | null; model?: string };
+  webSearch?: { backend?: SidecarBackend | null; model?: string; streamRoutedModelOutput?: boolean };
   vision?: { backend?: SidecarBackend | null; model?: string; reasoning?: VisionReasoning };
 }
 export interface ShadowCallData { enabled: boolean; model: string; sourceModels?: string[] }
@@ -151,13 +151,14 @@ export function updateJobLabel(status: UpdateJobStatus, t: (key: TKey) => string
 
 export function mergeSidecarSetting(
   current: SidecarSetting,
-  update?: { backend?: SidecarBackend | null; model?: string; reasoning?: VisionReasoning },
+  update?: { backend?: SidecarBackend | null; model?: string; reasoning?: VisionReasoning; streamRoutedModelOutput?: boolean },
 ): SidecarSetting {
   const merged = { ...current };
   if (update?.model !== undefined) merged.model = update.model;
   if (update?.backend === null) delete merged.backend;
   else if (update?.backend !== undefined) merged.backend = update.backend;
   if (update?.reasoning !== undefined) merged.reasoning = update.reasoning;
+  if (update?.streamRoutedModelOutput !== undefined) merged.streamRoutedModelOutput = update.streamRoutedModelOutput;
   return merged;
 }