Two runs on one thread_id at the same time do not interleave. The one that finishes last wins, the other's turn is gone, and both clients are told they succeeded — including a MESSAGES_SNAPSHOT that claims the loser's turn is the thread.
Reproduction
Two POST /runs against one threadId, launched together:
dA: RUN_FINISHED=True | its snapshot's user msgs: ['In one word: say APPLE']
dB: RUN_FINISHED=True | its snapshot's user msgs: ['In one word: say BANANA']
THREAD ACTUALLY HOLDS: ['In one word: say BANANA']
total messages: 2
APPLE's question and answer are not in the thread. Its client saw RUN_FINISHED and a snapshot asserting otherwise.
Against a real PostgreSQL checkpointer, on 0.7.0, with a real model.
Why the snapshot makes it worse
stream_turn reads the thread with aget_state at the start and the graph writes it back at the end, so this is a lost update. That much is inherent to the current model.
What #86 changed is the reporting: the closing MESSAGES_SNAPSHOT exists to tell a client "this is the thread as the server holds it". For the losing run that statement is false at the moment it is sent, and it is the one event a client is told to trust over its own copy. Before #86 a client could not detect the loss either, but nothing had promised it anything.
Not obviously the checkpointer's to fix
LangGraph's checkpointer does not lock a thread, and AsyncPostgresSaver writes per super-step rather than transactionally over a turn — so the two runs' writes interleave at the checkpoint level and the message reducer resolves to whichever wrote last.
Options, roughly in order of cost:
- Document it. A caller queues turns per thread. Cheapest, and honest — dss's
docs/API.md now says so. Does nothing for a second browser tab.
- Reject the overlap. Track in-flight
thread_ids in the process and answer the second run 409. Correct for one replica, wrong the moment there are two.
- Detect it. Compare the checkpoint id seen at run start with the one at write time and fail the turn that lost. Needs a checkpointer that reports it, but it makes the failure visible rather than silent.
- Serialise on the store. An advisory lock per thread. Real fix, real cost, and only for Postgres.
Worth deciding what the contract is before choosing: "one run per thread at a time, enforced" or "concurrent runs, last writer wins, and we say so".
Related
Two runs on one
thread_idat the same time do not interleave. The one that finishes last wins, the other's turn is gone, and both clients are told they succeeded — including aMESSAGES_SNAPSHOTthat claims the loser's turn is the thread.Reproduction
Two
POST /runsagainst onethreadId, launched together:APPLE's question and answer are not in the thread. Its client saw
RUN_FINISHEDand a snapshot asserting otherwise.Against a real PostgreSQL checkpointer, on 0.7.0, with a real model.
Why the snapshot makes it worse
stream_turnreads the thread withaget_stateat the start and the graph writes it back at the end, so this is a lost update. That much is inherent to the current model.What #86 changed is the reporting: the closing
MESSAGES_SNAPSHOTexists to tell a client "this is the thread as the server holds it". For the losing run that statement is false at the moment it is sent, and it is the one event a client is told to trust over its own copy. Before #86 a client could not detect the loss either, but nothing had promised it anything.Not obviously the checkpointer's to fix
LangGraph's checkpointer does not lock a thread, and
AsyncPostgresSaverwrites per super-step rather than transactionally over a turn — so the two runs' writes interleave at the checkpoint level and the message reducer resolves to whichever wrote last.Options, roughly in order of cost:
docs/API.mdnow says so. Does nothing for a second browser tab.thread_ids in the process and answer the second run409. Correct for one replica, wrong the moment there are two.Worth deciding what the contract is before choosing: "one run per thread at a time, enforced" or "concurrent runs, last writer wins, and we say so".
Related