| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1" /> |
| <title>Laguna Anticipatory Chat Demo</title> |
| <link rel="stylesheet" href="/static/styles.css" /> |
| </head> |
| <body> |
| <div id="authView" class="auth-view"> |
| <form id="authForm" class="auth-panel"> |
| <input |
| name="username" |
| type="hidden" |
| autocomplete="username" |
| value="demo" |
| /> |
| <label for="accessCode">Access code</label> |
| <div class="auth-row"> |
| <input |
| id="accessCode" |
| name="accessCode" |
| type="password" |
| autocomplete="current-password" |
| autofocus |
| /> |
| <button type="submit">Enter</button> |
| </div> |
| <p id="authError" class="error-line" aria-live="polite"></p> |
| </form> |
| </div> |
|
|
| <main id="appView" class="app hidden"> |
| <header class="app-header"> |
| <h1>Poor Man's Interaction Models</h1> |
| <p>by Overthinking Machines Lab</p> |
| </header> |
|
|
| <nav class="page-tabs" role="tablist" aria-label="Sections"> |
| <button |
| id="tabOverview" |
| class="page-tab active" |
| type="button" |
| role="tab" |
| aria-selected="true" |
| aria-controls="panelOverview" |
| data-tab="overview" |
| > |
| Overview |
| </button> |
| <button |
| id="tabReport" |
| class="page-tab" |
| type="button" |
| role="tab" |
| aria-selected="false" |
| aria-controls="panelReport" |
| data-tab="report" |
| > |
| Technical Report |
| </button> |
| <button |
| id="tabDemo" |
| class="page-tab" |
| type="button" |
| role="tab" |
| aria-selected="false" |
| aria-controls="panelDemo" |
| data-tab="demo" |
| > |
| Demo |
| </button> |
| </nav> |
|
|
| <section |
| id="panelOverview" |
| class="tab-panel" |
| role="tabpanel" |
| aria-labelledby="tabOverview" |
| data-panel="overview" |
| > |
| <section class="submission-context" aria-label="Submission context"> |
| <p class="submission-lede"> |
| Training Laguna XS.2 to (1) preemptively model two-side dialogue, |
| and (2) have a sense of time by using special tokens for micro-pauses: |
| we get pseudo-duplex interaction, including turn modelling, in a |
| text-only model. |
| </p> |
| <p> |
| Models: Laguna XS.2 (Non-interactive), with the interaction model |
| system prompt/harness but no training (Interactive), and with training |
| (Interactive RL). |
| </p> |
| </section> |
| </section> |
|
|
| <section |
| id="panelReport" |
| class="tab-panel hidden" |
| role="tabpanel" |
| aria-labelledby="tabReport" |
| data-panel="report" |
| > |
| <article class="technical-report"> |
| <h2>Pseudo-Duplex Interaction for Text-Only LLMs</h2> |
| <p class="report-byline">Overthinking Machines Lab</p> |
|
|
| <p> |
| Overthinking Machines Lab builds more interactive LLM systems for |
| the GPU poor. Here the target is narrow: take partial user input, |
| estimate whether the user is still speaking, and prepare a useful |
| reply before the user explicitly ends the turn. |
| </p> |
|
|
| <h3>Motivation</h3> |
| <p> |
| Most LLM interfaces are full-input-in, full-output-out. The user |
| finishes a turn, sends it, waits, and receives a complete response. |
| Human conversation is not like that. We hear partial utterances, |
| track pauses, predict likely continuations, and often know when a |
| turn is ending before the last word is fully processed. |
| </p> |
| <p> |
| Duplex speech models and richer interaction models attack this |
| problem directly. They are also harder to serve, tend to be more |
| expensive, and often complicate long-context text workflows. We |
| asked whether a useful part of this behavior can be moved into a |
| standard text LLM with a normal chat-completions serving path. |
| </p> |
|
|
| <h3>Problem Setup</h3> |
| <ul> |
| <li>Can a standard chat model produce a turn prediction and a response in one sequence?</li> |
| <li>Can turn completion be optimized jointly with response quality under partial input?</li> |
| <li>Can elapsed silence, represented as explicit tokens, improve this signal?</li> |
| </ul> |
|
|
| <h3>Modeling Task</h3> |
| <p> |
| We keep the same base interface (normal chat completion), and change only the |
| target structure. For each partial user turn, the model must output: |
| </p> |
| <pre><code><user_preemptive>[predicted remaining user text or empty]</user_preemptive> |
| <assistant_preemptive>[assistant continuation]</assistant_preemptive></code></pre> |
| <p> |
| An empty user segment means “no likely continuation”, and the system can |
| treat this as turn complete. Non-empty user continuation means the model |
| prefers waiting for more user content. |
| </p> |
|
|
| <h3>Time Representation</h3> |
| <p> |
| We inject a fixed micro-timing token in user input only: |
| <code><|silence|></code>. Each token maps to ~500ms of elapsed |
| pause after the current word boundary. The same lexical prefix therefore |
| appears with multiple temporal states. |
| </p> |
| <pre><code>user: i think we should <|silence|> <|silence|> |
| target: <user_preemptive>probably wait before answer</user_preemptive> |
| <assistant_preemptive>Let's hold until they finish.</assistant_preemptive></code></pre> |
| <pre><code>user: i think we should <|silence|> <|silence|> <|silence|> |
| target: <user_preemptive></user_preemptive> |
| <assistant_preemptive>I can pull up the options now.</assistant_preemptive></code></pre> |
|
|
| <h3>Dataset</h3> |
| <p> |
| We use real multi-turn voice transcripts with word timing metadata. |
| Silence tokens are added only in user turns to preserve system and |
| assistant speaker structure while preserving causality. |
| </p> |
| <p> |
| We augment each full session into many training states by truncating |
| user turns at different points: |
| </p> |
| <ul> |
| <li>mid-sentence and near punctuation boundaries</li> |
| <li>just after short and long intra-turn gaps</li> |
| <li>multiple positions in end-of-turn silence</li> |
| </ul> |
| <p> |
| The target always contains both the hidden user suffix and the eventual |
| assistant reply for the full reconstructed user turn. |
| </p> |
| <pre><code>History: |
| User: did you get a chance to look at the run |
| Assistant: yes, the reward moved but the eval was noisy |
|
|
| Current user prefix: |
| yeah i think <|silence|> <|silence|> |
|
|
| Target: |
| <user_preemptive>that is still enough for the demo</user_preemptive> |
| <assistant_preemptive>Then we should show the curve and be explicit that it is preliminary.</assistant_preemptive></code></pre> |
|
|
| <h3>Training</h3> |
| <p> |
| We train in a Prime RL loop over this tagged objective. The trainer |
| computes sequence loss on both tags and aligns reward with two goals: |
| </p> |
| <p> |
| 1) recover the missing user continuation, 2) generate the assistant |
| response for the true turn endpoint. Non-ideal tags (malformed or |
| partial tag blocks) are scored down. |
| </p> |
| <p> |
| We use curriculum scheduling over example generation. Early |
| training focuses on longer visible user text and short history. |
| Later phases reduce visible prefix, increase silence-only variants, |
| and increase conversation depth to test timing and turn prediction under |
| uncertainty. |
| </p> |
|
|
| <h3>Inference</h3> |
| <p> |
| The demo simulates streamed input by sending a new request on each |
| completed word boundary and then repeatedly after 500ms with extra |
| <code><|silence|></code> context. This produces multiple |
| candidate outputs over the same partial user text. |
| </p> |
| <p> |
| Silence tokens are retained in model history but removed from rendered |
| chat text to keep the interface readable. |
| </p> |
|
|
| <h3>Results</h3> |
| <p> |
| The first run is preliminary, but the training signal is clear. Reward |
| rises over the run, format adherence becomes reliable, and the model |
| improves on the specific behavior we care about most: predicting |
| whether the user turn is over. |
| </p> |
| <p> |
| Response-quality metrics are noisier, as expected for a small |
| conversational dataset with many acceptable replies. The stronger |
| signal is structural: the model learns the tags, uses silence as part |
| of the state, and improves turn-end prediction under both zero-history |
| and mixed-history evaluation. |
| </p> |
| <h4>Training Objective</h4> |
| <p> |
| The aggregate reward combines format, user-continuation prediction, |
| assistant response quality, and turn-boundary correctness. The curve |
| is noisy because each step samples different partial-turn states, but |
| the moving trend increases over the 50-step run. |
| </p> |
| <div class="report-figures report-figures-single" aria-label="Training reward plot"> |
| <figure> |
| <img src="/static/results/reward.png" alt="Training reward increasing over 50 steps" /> |
| <figcaption>Aggregate Prime RL reward over training.</figcaption> |
| </figure> |
| </div> |
| <p> |
| The reward distributions make the same movement easier to see. Early |
| checkpoints place most rollouts in the lowest reward bucket. By steps |
| 30 and 40, mass has shifted into the middle and high-reward buckets, |
| showing that improvement is not only a few lucky samples. |
| </p> |
| <div class="reward-distribution-strip" aria-label="Reward distribution snapshots across training"> |
| <figure> |
| <img src="/static/results/reward-distribution-step-0.png" alt="Reward distribution at step 0" /> |
| <figcaption>Step 0</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/reward-distribution-step-10.png" alt="Reward distribution at step 10" /> |
| <figcaption>Step 10</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/reward-distribution-step-20.png" alt="Reward distribution at step 20" /> |
| <figcaption>Step 20</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/reward-distribution-step-30.png" alt="Reward distribution at step 30" /> |
| <figcaption>Step 30</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/reward-distribution-step-40.png" alt="Reward distribution at step 40" /> |
| <figcaption>Step 40</figcaption> |
| </figure> |
| </div> |
|
|
| <h4>Interaction Mechanics</h4> |
| <p> |
| These metrics test whether the model learned the interaction protocol, |
| not just generic response text. Format reward measures whether the |
| model emits the two required tags. Turn-end correctness measures |
| whether empty versus non-empty <code><user_preemptive></code> |
| matches the target wait/answer decision. |
| </p> |
| <div class="report-figures" aria-label="Interaction mechanics plots"> |
| <figure> |
| <img src="/static/results/format-reward.png" alt="Format reward increasing toward 1.0" /> |
| <figcaption>Format reward: valid tagged output becomes reliable.</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/turn-end-correct.png" alt="Turn end correctness increasing over training" /> |
| <figcaption>Turn-end correctness: the model improves at deciding when to answer.</figcaption> |
| </figure> |
| </div> |
|
|
| <h4>Prediction Quality</h4> |
| <p> |
| These metrics are harder to optimize exactly because spoken replies |
| have many valid phrasings. User-completion similarity measures the |
| predicted continuation of the unfinished user turn. Assistant response |
| similarity and length similarity measure whether the eventual reply is |
| semantically and structurally close to the target reply. |
| </p> |
| <div class="report-figures" aria-label="Prediction quality plots"> |
| <figure> |
| <img src="/static/results/user-completion-similarity.png" alt="User completion similarity metric over training" /> |
| <figcaption>User-completion similarity: predicting the hidden suffix remains the hardest subtask.</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/assistant-response-similarity.png" alt="Assistant response similarity metric over training" /> |
| <figcaption>Assistant response similarity: noisy but improves late in training.</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/assistant-response-length-similarity.png" alt="Assistant response length similarity over training" /> |
| <figcaption>Assistant response length similarity: response shape starts to align with targets.</figcaption> |
| </figure> |
| </div> |
|
|
| <h4>Held-Out Evaluations</h4> |
| <p> |
| We evaluate with four rollouts per example. The zero-history easy set |
| isolates the base anticipatory task. The mixed-history set includes |
| conversation history and is closer to the demo setting. |
| </p> |
| <div class="report-figures" aria-label="Held-out evaluation plots"> |
| <figure> |
| <img src="/static/results/eval-zero-history-easy.png" alt="Zero history easy evaluation average increasing" /> |
| <figcaption>Zero-history easy eval: fast early improvement, then a smaller upward drift.</figcaption> |
| </figure> |
| <figure> |
| <img src="/static/results/eval-history-mixed.png" alt="Mixed history evaluation average increasing" /> |
| <figcaption>Mixed-history eval: improves despite longer context and more varied truncations.</figcaption> |
| </figure> |
| </div> |
| </article> |
| </section> |
|
|
| <section |
| id="panelDemo" |
| class="tab-panel hidden" |
| role="tabpanel" |
| aria-labelledby="tabDemo" |
| data-panel="demo" |
| > |
| <section class="demo-instructions" aria-label="Demo instructions"> |
| <p> |
| Intended for use with streaming STT and TTS for realtime interaction. |
| For the most realistic experience in this text-only demo, make spaces |
| or punctuation after words as soon as you've typed them, unless you're |
| done with your turn, in which case hit Enter. Refresh after changing |
| model. |
| </p> |
| </section> |
|
|
| <div class="mode-switch" role="group" aria-label="Mode"> |
| <button |
| class="mode-option" |
| type="button" |
| data-mode="baseline" |
| aria-pressed="false" |
| > |
| Non-interactive |
| </button> |
| <button |
| class="mode-option" |
| type="button" |
| data-mode="interactive-base" |
| aria-pressed="false" |
| > |
| Interactive |
| </button> |
| <button |
| class="mode-option active" |
| type="button" |
| data-mode="interactive-rl" |
| aria-pressed="true" |
| > |
| Interactive RL |
| </button> |
| </div> |
|
|
| <section class="workspace" aria-label="Anticipatory chat demo"> |
| <div id="history" class="history" aria-live="polite"></div> |
| <div class="composer-shell"> |
| <pre id="composerMirror" class="composer-mirror" aria-hidden="true"><span id="typedText"></span><span id="predictedText" class="predicted-text"></span><span id="thinking" class="thinking"></span></pre> |
| <textarea |
| id="composer" |
| class="composer-input" |
| rows="1" |
| spellcheck="false" |
| autocomplete="off" |
| autocapitalize="sentences" |
| aria-label="Message" |
| ></textarea> |
| </div> |
| <div id="assistantResponse" class="assistant-response" aria-live="polite"></div> |
| </section> |
|
|
| <aside class="calls" aria-label="Inference calls"> |
| <div class="calls-header"> |
| <div class="calls-title"> |
| <span>Inference calls</span> |
| <span id="latencyAverage" class="latency-average">avg ttft --</span> |
| <span id="accuracyAverage" class="accuracy-average">avg accuracy --</span> |
| </div> |
| <button id="clearCalls" type="button">Clear</button> |
| </div> |
| <div id="callList" class="call-list"></div> |
| </aside> |
| </section> |
|
|
| </main> |
|
|
| <script src="/static/app.js"></script> |
| </body> |
| </html> |
|
|
