Know when your client is still playing the audio you streamed to it.
English | 简体中文
A server streaming synthesized speech knows two things about every chunk it sends: when it sent it, and how long it plays. It never sees the client's speaker.
Yet plenty of decisions depend on knowing whether the client is still talking:
- Is this user speech an interruption, or a normal turn after the agent finished?
- If it is an interruption, where in the utterance did it land, so the agent can resume from there?
- Is it safe to send the next utterance yet?
The obvious estimate is wrong:
end := startedAt.Add(totalDuration) // wrongAudio is generated while it plays. As soon as one chunk is produced more slowly than the previous chunk takes to play, the client runs dry. Playback resumes only when the next chunk lands. That silence is in no chunk's duration — so the real end moves later, and because every stall shifts everything after it, the error accumulates.
sent: [A]----[B]------------[C]
| | |
naive: |AAAA|BBBB|CCCC| ends here ← too early
real: |AAAA| ..|BBBB| ....|CCCC| ends here
↑ ↑
stall stall ← silence nobody accounted for
playclock reconstructs the real timeline from the two facts the server actually has.
go get github.com/laconhub/playclockRequires Go 1.25+. No dependencies.
tl := playclock.New()
// as each chunk is handed to the client
tl.Append(time.Now(), chunkDuration)
// when the user starts speaking
if st := tl.State(); st.IsPlaying {
if st.Chunk != nil {
log.Printf("barge-in during chunk %d, %v in, %v left",
st.Chunk.Index, st.Offset, st.Remaining)
}
}
// between utterances
tl.Reset()The cursor advances chunk by chunk:
cursor = max(cursor, chunk.SentAt) + chunk.DurationTaking the later of "when the previous chunk finished" and "when this chunk arrived" reproduces each stall instead of ignoring it. That single max is the whole idea.
tl := playclock.New(playclock.WithTailBuffer(0))
tl.Append(start, 100*time.Millisecond) // plays 0..100ms
tl.Append(start.Add(300*time.Millisecond), 100*time.Millisecond) // arrives late
tl.EndsAt().Sub(start) // 400ms, not the naive 200msThe server cannot observe the last hop — network transit, then the client's own decode and playback buffering, sit between chunk sent and sound audible.
That delay is unmeasurable from the server. But its sign is known: the server's estimate is always early, never late. So the tail buffer extends the timeline forward only.
tl := playclock.New(playclock.WithTailBuffer(800 * time.Millisecond)) // defaultThe trade is direct:
| Tail buffer | Risk |
|---|---|
| Too small | the tail of an utterance is treated as finished while the client is still speaking it — a barge-in there is missed |
| Too large | ordinary speech after playback really ended is mistaken for an interruption |
Raise it for clients that are further away or buffer more aggressively.
Inside the tail buffer, playclock reports playback as ongoing but refuses to name a position:
st.IsPlaying // true
st.HasPosition() // false — st.Chunk is nilThis is deliberate, and it is the subtlest part of the design. Past the reconstructed end, the stream has most likely finished; a position would be a guess. Handing one out invites a caller to resume an utterance that already completed — for an LLM agent, that means replaying a sentence the user already heard.
The same applies mid-stall, where the client has run dry but more audio is already on the way.
So: check HasPosition() before using a position. Do not branch on IsPlaying alone.
The stalls the timeline finds are exactly the moments your generator fell behind realtime, so they come back as data:
for _, gap := range tl.Gaps() {
log.Printf("stalled %v after chunk %d", gap.Duration, gap.AfterIndex)
}
stats := tl.Stats()
// Chunks, Audio, Gaps, GapTime, StartedAt, EndsAt, Elapsed()GapTime / Audio is a direct stutter ratio. A rising value means synthesis is losing the race against playback — audible to the user as choppy speech, and worth alerting on.
| Method | Purpose |
|---|---|
New(opts...) |
create a timeline |
Append(sentAt, duration) |
record a chunk in order |
Insert(Chunk) |
record a chunk by explicit index, for out-of-order producers |
State() / StateAt(t) |
what the client is doing now / at an instant |
EndsAt() |
reconstructed end of playback |
Gaps() |
stalls where the client ran dry |
Stats() |
summary, including the stutter ratio inputs |
Reset() |
start a new stream |
Len() |
chunks recorded |
State carries IsPlaying, Chunk, Offset, Remaining, and the HasPosition() helper.
Options: WithTailBuffer(d), WithNowFunc(f).
A Timeline is safe for concurrent use — the goroutine streaming audio can record chunks while another asks what the client is doing.
Intervals are half-open. An instant landing exactly on a boundary belongs to what follows it, so EndsAt() is the first instant that is no longer playing. Chunk boundaries work the same way.
Chunks with a non-positive duration are ignored. Control frames and empty payloads occupy no audible time.
StateAt takes the instant as a parameter so a caller asking several questions about the same moment gets answers drawn from one point in time rather than several slightly different ones.
Apple M4 Pro, Go 1.25:
BenchmarkTimeline_Append-14 83.81 ns/op 0 allocs/op
BenchmarkTimeline_StateAt/chunks-10-14 138.3 ns/op 0 allocs/op
BenchmarkTimeline_StateAt/chunks-100-14 1180 ns/op 0 allocs/op
BenchmarkTimeline_EndsAt-14 618.5 ns/op 0 allocs/op
Reconstruction is a linear walk with no allocation. A spoken sentence is typically tens of chunks.
make test # race detector, shuffled order
make lint
make benchmarkWhy not just track playback on the client? If you can, do. This package is for when the client is a browser, a game engine, or a device you do not control the audio stack of — and all the server has is send times and durations.
Does this need the audio bytes? No. Only SentAt and Duration per chunk.
What if chunks arrive out of order? Use Insert with an explicit Index. Order matters: the timeline advances chunk by chunk, so a chunk in the wrong slot mixes up which stalls precede which audio.
Is it only for speech? No, but that is where it earns its keep. Any progressively generated audio stream has the same problem.
See CONTRIBUTING.md.