fix(api): hold the gmail sync cursor when message fetches fail

[ghost]

Why

The Gmail incremental sync can silently lose email. ingest() fetches each message in a batch of up to 120; when getMessage fails it does continue and the message is dropped. Back in incremental(), the history cursor is advanced to history.data.historyId whenever remaining === 0 — but remaining only counts messages beyond the batch cap, not failed fetches.

So a rate limit (429), an expired access token mid-batch (401), a timeout, or any transient 5xx makes the affected messages permanently absent: the cursor moves past them and they are never retried, even on later ticks.

What

• ingest() now returns a deferred count for message fetches that failed retryably (rate-limited, unauthorized, or failed with retryable: true).
• incremental() keeps the cursor at startHistoryId when deferred > 0, so the same history window is replayed next tick. Messages already written are filtered by gmailMessageId, so re-running the window is idempotent.
• Non-retryable outcomes still advance the cursor: a cursor-invalid (message deleted, 404/410) and a non-retryable failure (e.g. 403) will never succeed, so holding the cursor on them would stall the mailbox forever.

Tests

New apps/api/test/gmail-sync.spec.ts:

• rate-limited fetch → cursor held at startHistoryId
• retryable timeout → cursor held
• deleted message (cursor-invalid) → cursor advances
• non-retryable failure → cursor advances
• all fetches succeed → cursor advances

Verified: bun run check-types, bunx biome check on changed files, and the new spec all pass.

---

Summary by cubic

Prevent silent email loss by holding the Gmail sync cursor when message fetches fail with retryable errors. The same history window is replayed on the next tick; non-retryable cases still advance.

• Bug Fixes
  • ingest() tracks deferred for retryable outcomes (rate-limited, unauthorized, or failed with retryable: true).
  • incremental() keeps the cursor at startHistoryId when deferred > 0 and replays the window; stored messages are deduped by gmailMessageId.
  • Cursor advances on non-retryable outcomes (including cursor-invalid); log now includes deferred, and tests cover both paths.

^{Written for commit acfc9d0. Summary will update on new commits.}

[vercel]

Someone is attempting to deploy a commit to the Comp AI - PoC Team on Vercel.

A member of the Team first needs to authorize it.

[cubic-dev-ai]

2 issues found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/api/src/google/gmail-sync.service.ts">

<violation number="1" location="apps/api/src/google/gmail-sync.service.ts:247">
P1: A rate-limited or unauthorized message fetch is reported as a successful settle, so its backoff/reconnect state is never persisted and the next sync tick immediately replays up to 120 Gmail calls with the same bad condition. Preserve the cursor while routing these outcomes through the corresponding state transition (and return a rate-limited/reconnect outcome).</violation>

<violation number="2" location="apps/api/src/google/gmail-sync.service.ts:514">
P3: The new `isRetryable` type predicate is unsound: it is declared to narrow the result to the `failed` variant, but it returns `true` for `rate-limited` and `unauthorized` outcomes as well, which are different union members. It is safe only because the current caller uses it as a plain boolean; any future call that relies on the narrowed type would be handed the wrong shape. Consider dropping the bogus narrow guard and returning a plain `boolean`, or narrowing to the full union of retryable outcomes.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P1: A rate-limited or unauthorized message fetch is reported as a successful settle, so its backoff/reconnect state is never persisted and the next sync tick immediately replays up to 120 Gmail calls with the same bad condition. Preserve the cursor while routing these outcomes through the corresponding state transition (and return a rate-limited/reconnect outcome).

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At apps/api/src/google/gmail-sync.service.ts, line 247:

<comment>A rate-limited or unauthorized message fetch is reported as a successful settle, so its backoff/reconnect state is never persisted and the next sync tick immediately replays up to 120 Gmail calls with the same bad condition. Preserve the cursor while routing these outcomes through the corresponding state transition (and return a rate-limited/reconnect outcome).</comment>

<file context>
@@ -237,16 +239,20 @@ export class GmailSyncService {
 			const message = await this.gmail.getMessage(accessToken, id);
-			if (message.outcome !== "ok") continue;
+			if (message.outcome !== "ok") {
+				if (isRetryable(message)) deferred += 1;
+				continue;
+			}
</file context>

[cubic-dev-ai]

P3: The new isRetryable type predicate is unsound: it is declared to narrow the result to the failed variant, but it returns true for rate-limited and unauthorized outcomes as well, which are different union members. It is safe only because the current caller uses it as a plain boolean; any future call that relies on the narrowed type would be handed the wrong shape. Consider dropping the bogus narrow guard and returning a plain boolean, or narrowing to the full union of retryable outcomes.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At apps/api/src/google/gmail-sync.service.ts, line 514:

<comment>The new `isRetryable` type predicate is unsound: it is declared to narrow the result to the `failed` variant, but it returns `true` for `rate-limited` and `unauthorized` outcomes as well, which are different union members. It is safe only because the current caller uses it as a plain boolean; any future call that relies on the narrowed type would be handed the wrong shape. Consider dropping the bogus narrow guard and returning a plain `boolean`, or narrowing to the full union of retryable outcomes.</comment>

<file context>
@@ -502,3 +508,10 @@ export class GmailSyncService {
+
+function isRetryable(
+	result: GoogleResult<unknown>,
+): result is Extract<GoogleResult<unknown>, { outcome: "failed" }> {
+	if (result.outcome === "failed") return result.retryable;
+	return result.outcome === "rate-limited" || result.outcome === "unauthorized";
</file context>

Suggested change

	): result is Extract<GoogleResult<unknown>, { outcome: "failed" }> {
	): boolean {