Skip to content

sessions: the change-feed cache mirror is duplicated in every host #2205

Description

@mattzcarey

Every host that keeps an in-memory transcript reimplements the same reduction over sessions.subscribe(). Think and AIChatAgent each carry their own copy today, and pi will need a third.

The duplication

Both hosts write the same switch:

event what both do
append find by id; replace if present, push if not
update find by id; replace in place, ignore if absent
delete filter out the removed ids
clear empty the array

plus their own findIndex-based upsert and patch helpers around it. Think._upsertCachedMessage / _patchCachedMessage / _replaceCachedMessages and the equivalent inline blocks in AIChatAgent are the same code with different field names.

This is a reduction over an ordered change feed, which is exactly the kind of thing the capability that emits the feed should be able to do for you.

Why it has not moved yet

The differences between the two are real, not incidental, and a naive shared implementation would be wrong for both:

  • Think resyncs where AIChat patches. A branch append (event.parentId !== undefined) and a compact event both make Think do a full _syncMessages(), because the path itself changed and an in-place patch cannot express that. It also refreshes its system prompt on compact. AIChat has no branch model and no prompt to refresh.
  • AIChat's cache is public API. this.messages is a documented field that existing subclasses assign to directly. It cannot become an opaque handle without a breaking change.
  • Coherence during streaming is load-bearing. The comments around _applyToolUpdateToMessages record a real bug — a full re-read mid-turn drops in-flight messages whose parent chain is not yet persisted — and the fix was specifically to patch in place rather than resync. Any shared mirror has to preserve that, and getting it wrong reintroduces a bug that took two reverts to settle.

So the shape is not simply "move the switch down". It needs a seam that lets a host choose patch-vs-resync per event, and lets AIChat keep a plain array it owns.

Sketch

Something like a session.mirror(target, hooks) that owns the reduction and calls back for the decisions only the host can make:

const mirror = session.mirror({
  onResyncNeeded: () => this._syncMessages(),   // Think: branch append, compact
  transform: (m) => autoTransformMessages([m])[0]  // AIChat: v4 -> v5
})

The transform hook matters: AIChat's #messageForCache does version transformation, which is genuinely its own concern and must stay a host decision rather than something sessions knows about.

Prior art in this PR

#2196 already removed one instance of hosts working around the feed. appendMessage used to return getMessageRaw on its duplicate paths, so the feed emitted pointer-form messages on some appends and inline messages on others. Think compensated by serializing every incoming message and substring-searching it for attachment:sha256: before deciding whether to re-read — on the streaming hot path. Making the feed emit one consistent shape deleted that helper outright.

That is the same category of problem: when the feed's contract is not quite enough, every host grows its own patch. Worth doing the mirror properly rather than waiting for a third copy to appear in pi.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions