[superlog] Log unknown custom tool names and defer failure to allow model self-correction

[superlog-app]

Summary

When the investigation agent calls a tool name the runtime doesn't recognize (a hallucinated or mistyped tool name), the worker was failing the run immediately — before the model had a chance to see the error response that was just sent back to it in the same collect pass. This meant investigations that ran for 15+ minutes could be permanently aborted on a single naming mistake.

Root cause

syncRunningAgentRun checked snapshot.unknownCustomTools.length > 0 and called failAgentRun before the shouldDeferSteering guard. Because collectManagedAgentRun sends the error acks for unknown tools in the same pass that populates snapshot.unknownCustomTools, the model had not yet received the error response ("unknown tool: X") at the moment the run was killed. On the very next tick, snapshot.unknownCustomTools would have been empty (the tool is now acked in the stream) and the run would have continued normally — but it never got that tick.

Additionally, the specific tool names were stored only in the DB failure summary, not in any structured log field, making the comment "so we can audit them later" impossible to satisfy from logs alone.

Remediation

Two changes, both in apps/worker/src/agent-runs/sync.ts:

1. Move the check after the DB update — cumulativeRuntimeMinutes and lastSyncedAt are now current before we touch this path.

2. Emit a WARN with the tool names, then defer instead of failing — when sentToolAckCount > 0 (always true for unacked unknown tools), the error ack was sent this pass and the model will see it on its next event. We return early (before steering/completion) and let the model self-correct. Persistent hallucination is caught by the existing wall-clock budget. The only code path that still calls failAgentRun is the sentToolAckCount === 0 edge case, which shouldn't occur in practice but guards against unexpected regressions.

The structured unknown_tool_names field in the WARN makes tool-name auditing possible directly from log queries without a DB lookup.

Incident on Superlog

---

Was this PR helpful? Leave feedback — goes straight to the Superlog team.

---

Summary by cubic

Prevents premature run failures when a model calls unknown custom tools by logging the tool names and deferring failure so the model can self-correct on the next tick.

• Bug Fixes
  • Moved the unknown-tool check after the DB update to keep runtime metrics current.
  • When acks were sent this pass, emit a WARN with unknown_tool_names and return early to let the model self-correct.
  • Fail immediately only if no acks were sent (guard path).

^{Written for commit 05e541e. Summary will update on new commits.}