Good point, and agreed. Tagging by arrival time only protects the local queue; it can't tell a stale upstream chunk from new speech. The proxy relays a single ordered upstream stream, so it bumps the generation the moment it sees interrupted and tags each binary frame as it arrives after that, but that still relies on the upstream not emitting old-turn audio after its own interrupted event. We'll document the two separately when the counter lands: local queue protection on the client, and the upstream ordering assumption on the proxy. Thanks for pushing on this, it made the design clearer.
Veda Labs PRO
AI & ML interests
Recent Activity
Organizations
Good question, and it exposes a weakness in what I suggested last time, so let me correct it.
In our proxy, turn is not the close of the interrupted response. Once interrupted arrives, the cut response is already over, and the next turn marks completion of the response that follows. So gating playback until turn would discard valid new speech. Please drop that part of my earlier suggestion.
Without a response ID on PCM chunks, the honest boundary is interrupted alone: clear the scheduled queue at that moment and treat binary arriving after it as the new response. If you need a hard guarantee in a fork, the clean fix is on the proxy side: keep a generation counter, increment it on interrupted, and tag each outgoing binary frame (for example with a small header byte) so the client can drop any frame from an older generation. That's on our list for the Space too.
Thanks for pushing on this. It's exactly the kind of detail that makes barge-in feel right.
Vedika Live: Realtime Voice on a Hugging Face Space
If you want to poke at it live: https://huggingface.co/spaces/Veda-Labs/vedika-demo β allow mic, pick a voice, Start Live, and try barge-in mid-sentence. Would love to hear how it feels on your side.
Great catch β you're reading the pipe correctly.
interrupted is forwarded straight from Live server_content.interrupted. The client only clears already-scheduled Web Audio sources so barge-in feels instant. Binary PCM after that is still accepted and played, because those frames are the next Live response audio on the same socket.
Ordering guarantee: Live marks the cut with interrupted, then streams the new turn's audio. We also forward turn_complete as a turn event for a hard boundary. The Space proxy does not stamp a generation id on binary frames today, so the clear turn boundary is that JSON pair (interrupted β later turn), not a per-chunk response id. In practice Live stops emitting the interrupted turn's audio after the interrupt signal; if a rare late chunk from the old turn slips through, queue clear already dropped anything scheduled, and the next audible audio is the new response.
If you're hardening a fork, a cheap upgrade is to ignore binary until the first post-interrupt chunk (or until turn) β happy to hear how that lands for you.
Thank you so much for reading β and for the thoughtful note. You're right that duplex buffering and transport chunking are the fiddly parts; we kept that work on the Space side so the browser stays thin. Really glad the sub-500ms feel came through. If you try the demo, we'd love to hear how it lands for you.