Add GitHub Pages demo and recording functionality

- Introduced a new demo section in README.md detailing how to deploy a static page on GitHub Pages using a recorded session.
- Added a GitHub Actions workflow for automatic deployment of the demo to GitHub Pages.
- Created a script to generate a demo recording (`scripts/make_demo_recording.py`) that simulates engine events for the demo.
- Updated the engine to support event recording to a JSONL file for replay functionality.
- Enhanced the web client to handle live and replay modes, including UI updates for replay controls and connection status.
- Added styles and elements in the web interface to support the new replay features.
- Updated configuration to allow specifying a recording path for event logging.
- Included a new `web/events.jsonl` file to store demo events for playback.
- Adjusted server arguments to enable recording mode.
- Improved event emission to support recording of engine events for the demo.
- Enhanced README with instructions for using the demo and generating recordings.

.claude/settings.local.json

@@ -6,7 +6,9 @@
       "Bash(python -m pytest tests/test_block_manager.py tests/test_scheduler.py -v)",
       "Bash(pip install *)",
       "Bash(python -m pytest tests/ -v)",
-      "Bash(python -c ' *)"
+      "Bash(python -c ' *)",
+      "Bash(python scripts/make_demo_recording.py)",
+      "Bash(python *)"
     ]
   }
 }

.github/workflows/deploy-pages.yml

@@ -0,0 +1,47 @@
+name: Deploy demo to GitHub Pages
+
+# Publishes the static visualization from web/ to GitHub Pages on every push
+# to main (and on demand from the Actions tab).  In Pages mode it runs against
+# a committed web/events.jsonl recording — see the README for how to capture
+# a fresh one.
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "web/**"
+      - ".github/workflows/deploy-pages.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow only one Pages deployment at a time; skip queued ones if a newer
+# commit arrives.
+concurrency:
+  group: pages
+  cancel-in-progress: true
+
+jobs:
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Pages
+        uses: actions/configure-pages@v5
+
+      - name: Upload web/ as artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: web
+
+      - name: Deploy
+        id: deployment
+        uses: actions/deploy-pages@v4

README.md

@@ -76,6 +76,33 @@ pip install pytest
 python -m pytest tests/
 ```
 
+## GitHub Pages demo (replay mode)
+
+The visualization can run as a **static page** on GitHub Pages with no
+backend.  It plays back a recorded session from `web/events.jsonl`:
+
+1. The repo ships a fabricated `web/events.jsonl` so the page works on first
+   deploy (run `python scripts/make_demo_recording.py > web/events.jsonl` to
+   regenerate).
+2. To use a **real** recording instead, run the server with `--record`:
+   ```bash
+   python -m tiny_vllm.server --record web/events.jsonl
+   # …submit some prompts via the UI or smoke_client…
+   # Ctrl-C the server.  events.jsonl now contains the full session.
+   git add web/events.jsonl && git commit -m "fresh demo recording" && git push
+   ```
+3. Enable Pages once: **repo → Settings → Pages → Source: "GitHub Actions"**.
+   The workflow in `.github/workflows/deploy-pages.yml` then publishes
+   `web/` on every push to `main` that touches it.
+
+The page auto-detects mode:
+- Tries `/engine/events` SSE first; if it responds within 2s it's **live**.
+- Otherwise falls back to **replay**, fetching `events.jsonl` from the same
+  directory and playing it back with original timing (speed control / pause
+  / restart in the controls row).
+- Force a mode with `?mode=replay` or `?mode=live`; point at a different
+  recording with `?session=URL`.
+
 ## What the demo page shows
 
 | Panel | What you're looking at |

scripts/make_demo_recording.py

@@ -0,0 +1,261 @@
+"""Fabricate a representative events.jsonl for the GH Pages demo.
+
+We don't need a model or torch here — we just synthesize a sequence of engine
+events that exercises:
+
+  * a prompt being admitted (request event + first step admitting it)
+  * chunked prefill across two steps
+  * decoding several tokens
+  * a second request that hits the prefix cache on identical prefix tokens
+  * a finished sequence and a freed-but-cached block
+
+The output is read by `web/app.js`'s replay mode.
+
+Run:
+    python scripts/make_demo_recording.py > web/events.jsonl
+"""
+from __future__ import annotations
+
+import json
+import sys
+from copy import deepcopy
+
+NUM_BLOCKS = 24
+BLOCK_SIZE = 8
+MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
+
+CONFIG = {
+    "model": MODEL,
+    "block_size": BLOCK_SIZE,
+    "num_blocks": NUM_BLOCKS,
+    "max_num_seqs": 8,
+    "max_num_batched_tokens": 32,
+    "prefix_caching": True,
+}
+
+# State we evolve.
+ref = [0] * NUM_BLOCKS
+hashed = [False] * NUM_BLOCKS
+lookups = 0
+hits = 0
+
+events: list[dict] = []
+ts = 0.0
+step = 0
+
+
+def pool_snapshot() -> dict:
+    return {
+        "num_blocks": NUM_BLOCKS,
+        "block_size": BLOCK_SIZE,
+        "num_free_blocks": sum(1 for r in ref if r == 0),
+        "num_cached_entries": sum(1 for h in hashed if h),
+        "prefix_cache_hits": hits,
+        "prefix_cache_lookups": lookups,
+        "ref_counts": list(ref),
+        "hashed": list(hashed),
+    }
+
+
+def seq_view(rid, sid, status, prompt_len, num_gen, num_computed, num_cached, block_table) -> dict:
+    return {
+        "seq_id": sid, "request_id": rid, "status": status,
+        "prompt_len": prompt_len, "num_generated": num_gen,
+        "num_computed_tokens": num_computed,
+        "num_cached_prefix_tokens": num_cached,
+        "block_table": list(block_table),
+    }
+
+
+def emit(ev_type: str, payload: dict) -> None:
+    global ts
+    events.append({"type": ev_type, "step": step, "timestamp": round(ts, 3), "payload": payload})
+
+
+def emit_snapshot(running, waiting) -> None:
+    emit("snapshot", {
+        "step": step,
+        "config": CONFIG,
+        "block_pool": pool_snapshot(),
+        "running": running,
+        "waiting": waiting,
+    })
+
+
+def emit_step(dur_ms, n_tok, n_pf, n_dec, deltas, running, waiting,
+              admitted=None, finished=None, preempted=None):
+    emit("step", {
+        "duration_ms": dur_ms,
+        "num_tokens": n_tok,
+        "num_seqs": n_pf + n_dec,
+        "num_prefill_seqs": n_pf,
+        "num_decode_seqs": n_dec,
+        "deltas": deltas or [],
+        "newly_admitted": admitted or [],
+        "finished": finished or [],
+        "preempted": preempted or [],
+        "snapshot": {
+            "step": step,
+            "config": CONFIG,
+            "block_pool": pool_snapshot(),
+            "running": running,
+            "waiting": waiting,
+        },
+    })
+
+
+# ---------- script the session ----------
+
+# Initial empty snapshot.
+emit_snapshot(running=[], waiting=[])
+ts += 0.4
+
+# --- request A admitted with a 20-token prompt --------------------------
+RID_A = "demo-aaaa1111"
+SID_A = 1
+PROMPT_A = "Explain paged attention in two sentences. Then explain prefix caching."
+
+step = 1
+ts += 0.1
+emit("request", {"request_id": RID_A, "seq_id": SID_A, "prompt": PROMPT_A,
+                 "prompt_len": 20, "max_tokens": 24})
+
+# Step 1: chunked prefill, first chunk (16 tokens → 2 blocks).
+ref[0] = ref[1] = 1
+hashed[0] = hashed[1] = True
+ts += 0.05
+running_A_pf1 = [seq_view(RID_A, SID_A, "prefilling", 20, 0, 16, 0, [0, 1])]
+emit_step(280, 16, 1, 0, [], [], [running_A_pf1[0]], admitted=[SID_A])
+# (waiting moved to running at end of prefill; here still prefilling, so it
+# stays in waiting in our scheduler; emit fix:)
+events[-1]["payload"]["snapshot"]["running"] = []
+events[-1]["payload"]["snapshot"]["waiting"] = running_A_pf1
+ts += 0.3
+
+# Step 2: finish prefill (4 more tokens). Need 1 more block.
+step = 2
+ref[2] = 1  # block 2 holds last 4 tokens (partial)
+ts += 0.05
+running_A = [seq_view(RID_A, SID_A, "running", 20, 0, 20, 0, [0, 1, 2])]
+emit_step(180, 4, 1, 0, [], running_A, [])
+ts += 0.2
+
+# Steps 3-7: A decodes 5 tokens. Each step appends one slot; block 2 fills
+# at token 24, then block 3 starts.
+TOKS_A = [" Paged", " attention", " splits", " keys", " and"]
+for i, tok in enumerate(TOKS_A):
+    step += 1
+    ts += 0.12
+    n_gen = i + 1
+    n_comp = 20 + i  # after fwd: num_computed_tokens increments by 1 each decode
+    # Block table grows when crossing boundary.
+    bt = [0, 1, 2]
+    if 20 + n_gen > 24:
+        bt = [0, 1, 2, 3]
+        ref[3] = 1
+    # Maybe hash block 2 when it just filled (after sampling tok at pos 23 → 24 tokens computed).
+    if 20 + n_gen >= 24 and not hashed[2]:
+        hashed[2] = True
+    running_A = [seq_view(RID_A, SID_A, "running", 20, n_gen, 20 + n_gen,
+                          0, bt)]
+    emit_step(95, 1, 0, 1,
+              [{"request_id": RID_A, "new_text": tok, "finished": False, "finish_reason": None}],
+              running_A, [])
+
+# --- request B admitted, identical prefix → cache hit -------------------
+step += 1
+RID_B = "demo-bbbb2222"
+SID_B = 2
+PROMPT_B = PROMPT_A   # same → prefix cache hit
+ts += 0.6
+emit("request", {"request_id": RID_B, "seq_id": SID_B, "prompt": PROMPT_B,
+                 "prompt_len": 20, "max_tokens": 24})
+
+step += 1
+ts += 0.05
+# Hit lookups: 2 full blocks of B's prompt (block_size=8 → 16/8=2 blocks).
+# Both hit because A's first 2 blocks are hashed (and currently in use).
+lookups += 2
+hits += 2
+ref[0] += 1
+ref[1] += 1
+# B needs a 3rd block (partial, 4 tokens). Fresh: block 4.
+ref[4] = 1
+running_B_pf = [seq_view(RID_B, SID_B, "prefilling", 20, 0, 16, 16, [0, 1, 4])]
+# A is still running and decoding in this step.
+step_A_n_gen = len(TOKS_A)
+running_A_now = [seq_view(RID_A, SID_A, "running", 20, step_A_n_gen,
+                          20 + step_A_n_gen, 0, [0, 1, 2, 3])]
+emit_step(140, 5, 1, 1,
+          [{"request_id": RID_A, "new_text": " values", "finished": False, "finish_reason": None}],
+          [running_A_now[0]], [running_B_pf[0]], admitted=[SID_B])
+ts += 0.12
+
+# Step: B finishes prefill (4 tokens), A decodes one more.
+step += 1
+running_A_now = [seq_view(RID_A, SID_A, "running", 20, step_A_n_gen + 1,
+                          21 + step_A_n_gen, 0, [0, 1, 2, 3])]
+running_B = [seq_view(RID_B, SID_B, "running", 20, 0, 20, 16, [0, 1, 4])]
+emit_step(110, 5, 1, 1,
+          [{"request_id": RID_A, "new_text": " into", "finished": False, "finish_reason": None}],
+          [running_A_now[0], running_B[0]], [])
+ts += 0.12
+
+# Steps: both decode in lock-step for a few rounds.
+A_tokens = [" small", ",", " fixed", "-size", " blocks", " stored", " in"]
+B_tokens = [" Prefix", " caching", " reuses", " those", " blocks", " across", " requests"]
+for i in range(7):
+    step += 1
+    ts += 0.10
+    a_gen = step_A_n_gen + 2 + i
+    b_gen = i + 1
+    a_bt = [0, 1, 2, 3]
+    b_bt = [0, 1, 4]
+    # B crosses block boundary at b_gen such that 20 + b_gen > 24 → needs block 5.
+    if 20 + b_gen > 24 and ref[5] == 0:
+        ref[5] = 1
+    if 20 + b_gen > 24:
+        b_bt = [0, 1, 4, 5]
+    if 20 + b_gen >= 24 and not hashed[4]:
+        hashed[4] = True
+    seq_a = seq_view(RID_A, SID_A, "running", 20, a_gen, 20 + a_gen, 0, a_bt)
+    seq_b = seq_view(RID_B, SID_B, "running", 20, b_gen, 20 + b_gen, 16, b_bt)
+    emit_step(105, 2, 0, 2,
+              [{"request_id": RID_A, "new_text": A_tokens[i], "finished": False, "finish_reason": None},
+               {"request_id": RID_B, "new_text": B_tokens[i], "finished": False, "finish_reason": None}],
+              [seq_a, seq_b], [])
+
+# A finishes (hits EOS or max_tokens).
+step += 1
+ts += 0.10
+ref[0] -= 1   # A releases its blocks; block 0,1 still held by B so refcount drops to 1
+ref[1] -= 1
+ref[2] = 0    # block 2 (A only) → goes to cached free list (hashed)
+ref[3] = 0    # block 3 (A only) → if hashed, cached; else uncached free
+# Block 2 was hashed earlier; block 3 we never hashed (partial in A's decode). For demo,
+# hash block 3 to make the cache view richer.
+hashed[3] = True
+seq_b = seq_view(RID_B, SID_B, "running", 20, 8, 28, 16, [0, 1, 4, 5])
+emit_step(85, 1, 0, 1,
+          [{"request_id": RID_A, "new_text": " the GPU.", "finished": True, "finish_reason": "stop"}],
+          [seq_b], [], finished=[RID_A])
+
+# B finishes too.
+step += 1
+ts += 0.10
+ref[0] -= 1
+ref[1] -= 1
+ref[4] = 0
+ref[5] = 0
+hashed[5] = True
+emit_step(80, 1, 0, 1,
+          [{"request_id": RID_B, "new_text": " for the same prompts.", "finished": True, "finish_reason": "stop"}],
+          [], [], finished=[RID_B])
+
+
+# ---------- emit ----------
+
+out = sys.stdout
+for ev in events:
+    out.write(json.dumps(ev, separators=(",", ":")))
+    out.write("\n")

tiny_vllm/config.py

@@ -25,6 +25,8 @@ class EngineConfig:
     # Logging / events
     emit_events: bool = True              # produce engine events for the UI
     event_buffer: int = 256
+    record_path: Optional[str] = None     # JSONL file to append every event to
+                                          # (powers the static GH-Pages replay)
 
     def __post_init__(self) -> None:
         if self.max_num_batched_tokens < self.block_size:

tiny_vllm/engine.py

@@ -16,11 +16,12 @@ from __future__ import annotations
 
 import asyncio
 import itertools
+import json
 import time
 import uuid
 from collections import deque
 from dataclasses import dataclass, field
-from typing import AsyncIterator, Optional
+from typing import AsyncIterator, Optional, TextIO
 
 from .block_manager import BlockManager
 from .config import EngineConfig, SamplingParams
@@ -69,6 +70,9 @@ class LLMEngine:
         self._step_idx = 0
         self._run_task: Optional[asyncio.Task] = None
         self._wake = asyncio.Event()
+        # recording (for the static GH-Pages replay)
+        self._record_fh: Optional[TextIO] = None
+        self._record_t0: float = 0.0
 
     # ---- lifecycle ------------------------------------------------------
 
@@ -87,6 +91,19 @@ class LLMEngine:
         )
         self.scheduler = Scheduler(self.config, self.block_manager)
         self.sampler = Sampler(self.model_runner.device)
+
+        # Open the recorder *after* the block manager exists so the initial
+        # snapshot we write is valid.
+        if self.config.record_path:
+            self._record_fh = open(self.config.record_path, "w", buffering=1)
+            self._record_t0 = time.monotonic()
+            self._record({
+                "type": "snapshot",
+                "step": 0,
+                "timestamp": 0.0,
+                "payload": self.snapshot(),
+            })
+
         self._run_task = asyncio.create_task(self._run_loop())
 
     async def shutdown(self) -> None:
@@ -97,6 +114,12 @@ class LLMEngine:
                 await asyncio.wait_for(self._run_task, timeout=5)
             except asyncio.TimeoutError:
                 self._run_task.cancel()
+        if self._record_fh is not None:
+            try:
+                self._record_fh.close()
+            except Exception:
+                pass
+            self._record_fh = None
 
     # ---- request submission --------------------------------------------
 
@@ -110,8 +133,10 @@ class LLMEngine:
             raise RuntimeError("engine not started")
         if isinstance(prompt, str):
             token_ids = self.model_runner.encode(prompt)
+            prompt_text = prompt
         else:
             token_ids = list(prompt)
+            prompt_text = self.model_runner.decode(token_ids)
         if not token_ids:
             raise ValueError("empty prompt")
         if len(token_ids) >= self.config.max_model_len:
@@ -129,6 +154,13 @@ class LLMEngine:
         self._prev_text_len[rid] = 0
         assert self.scheduler is not None
         self.scheduler.add(seq)
+        self._emit("request", {
+            "request_id": rid,
+            "seq_id": seq.seq_id,
+            "prompt": prompt_text,
+            "prompt_len": len(token_ids),
+            "max_tokens": sampling_params.max_tokens,
+        })
         self._wake.set()
         return rid
 
@@ -166,7 +198,7 @@ class LLMEngine:
             pass
 
     def _emit(self, event_type: str, payload: dict) -> None:
-        if not self.config.emit_events or not self._event_subscribers:
+        if not self.config.emit_events:
             return
         ev = EngineEvent(
             step=self._step_idx,
@@ -178,7 +210,6 @@ class LLMEngine:
             try:
                 q.put_nowait(ev)
             except asyncio.QueueFull:
-                # Drop oldest, push new.
                 try:
                     q.get_nowait()
                 except asyncio.QueueEmpty:
@@ -187,6 +218,23 @@ class LLMEngine:
                     q.put_nowait(ev)
                 except asyncio.QueueFull:
                     pass
+        # Mirror into the on-disk recording (timestamps re-based to t0).
+        if self._record_fh is not None:
+            self._record({
+                "type": ev.type,
+                "step": ev.step,
+                "timestamp": ev.timestamp - self._record_t0,
+                "payload": ev.payload,
+            })
+
+    def _record(self, ev: dict) -> None:
+        fh = self._record_fh
+        if fh is None:
+            return
+        try:
+            fh.write(json.dumps(ev, separators=(",", ":")) + "\n")
+        except Exception:
+            pass
 
     # ---- inspection ----------------------------------------------------
 
@@ -301,7 +349,9 @@ class LLMEngine:
                     self.scheduler.running.remove(seq)
                 self.block_manager.free(seq)
 
-            # Emit outputs to per-request queues.
+            # Emit outputs to per-request queues, and collect per-step deltas
+            # for the event stream (powers the replay UI's text panes).
+            step_deltas: list[dict] = []
             for item in sched.scheduled:
                 seq = item.seq
                 rid = seq.request_id
@@ -325,8 +375,14 @@ class LLMEngine:
                     q = self._output_queues.get(rid)
                     if q is not None:
                         await q.put(si)
+                    if new_text or is_done:
+                        step_deltas.append({
+                            "request_id": rid,
+                            "new_text": new_text,
+                            "finished": is_done,
+                            "finish_reason": seq.finish_reason,
+                        })
                     if is_done:
-                        # Clean up.
                         self._sequences.pop(rid, None)
                         self._prev_text_len.pop(rid, None)
 
@@ -340,6 +396,7 @@ class LLMEngine:
                 "preempted": sched.preempted,
                 "newly_admitted": sched.newly_admitted,
                 "finished": [s.request_id for s in finished_now],
+                "deltas": step_deltas,
                 "snapshot": self.snapshot(),
             })
 

tiny_vllm/server.py

@@ -282,6 +282,9 @@ def main() -> None:
     parser.add_argument("--max-num-batched-tokens", type=int, default=512)
     parser.add_argument("--max-model-len", type=int, default=2048)
     parser.add_argument("--disable-prefix-caching", action="store_true")
+    parser.add_argument("--record", default=None,
+                        help="Append every engine event to this JSONL file "
+                             "(e.g. web/events.jsonl) to power the static replay demo.")
     parser.add_argument("--host", default="0.0.0.0")
     parser.add_argument("--port", type=int, default=8000)
     args = parser.parse_args()
@@ -296,6 +299,7 @@ def main() -> None:
         max_num_batched_tokens=args.max_num_batched_tokens,
         max_model_len=args.max_model_len,
         enable_prefix_caching=not args.disable_prefix_caching,
+        record_path=args.record,
     )
 
     import uvicorn

web/app.js

@@ -1,12 +1,16 @@
 /* tiny_vllm — demo page client.
  *
- * Two streams in play:
+ * Runs in one of two modes:
  *
- *   /engine/events    — engine state snapshots (one per scheduling step)
- *   /generate         — token-level deltas for whatever prompt this page sent
+ *   LIVE     — talks to a tiny_vllm server.  Subscribes to /engine/events
+ *              (SSE) and POSTs to /generate to submit prompts.
+ *   REPLAY   — no backend.  Fetches a pre-recorded events.jsonl from the
+ *              same directory and dispatches each event with original timing.
+ *              Used for the GitHub Pages demo.
  *
- * The page itself is stateless; everything is driven by what comes off the
- * event stream.  Token deltas from /generate are merged into per-request UI.
+ * Mode is auto-detected: we try SSE first; if there's no response within a
+ * short window we fall back to replay.  Force a mode with ?mode=replay or
+ * ?mode=live in the URL.  Point at a different recording with ?session=URL.
  */
 
 const $ = (id) => document.getElementById(id);
@@ -27,6 +31,11 @@ const ui = {
   seqs: $("seqs"),
   send: $("send"),
   sendTwice: $("send-twice"),
+  prompt: $("prompt"),
+  banner: $("banner"),
+  speed: $("speed"),
+  playPause: $("play-pause"),
+  restart: $("restart"),
 };
 
 const state = {
@@ -38,6 +47,8 @@ const state = {
   requests: new Map(),
   // seq_id -> { request_id, blockTable, cachedPrefixBlocks, status, ... }
   seqsBySeqId: new Map(),
+  mode: "connecting",     // "live" | "replay" | "connecting"
+  replay: null,           // controller object for replay mode
 };
 
 function logLine(html, cls = "") {
@@ -46,6 +57,46 @@ function logLine(html, cls = "") {
   ui.log.scrollTop = ui.log.scrollHeight;
 }
 
+function setBanner(text, cls) {
+  if (!ui.banner) return;
+  ui.banner.textContent = text;
+  ui.banner.className = `banner ${cls || ""}`;
+  ui.banner.style.display = text ? "" : "none";
+}
+
+function setMode(mode) {
+  state.mode = mode;
+  if (mode === "live") {
+    ui.connection.textContent = "live";
+    ui.connection.classList.remove("offline");
+    ui.connection.classList.add("online");
+    ui.send.disabled = false;
+    ui.sendTwice.disabled = false;
+    ui.prompt.disabled = false;
+    setBanner("", "");
+    if (ui.speed) ui.speed.style.display = "none";
+    if (ui.playPause) ui.playPause.style.display = "none";
+    if (ui.restart) ui.restart.style.display = "none";
+  } else if (mode === "replay") {
+    ui.connection.textContent = "replay";
+    ui.connection.classList.remove("offline");
+    ui.connection.classList.add("replay");
+    ui.send.disabled = true;
+    ui.sendTwice.disabled = true;
+    ui.prompt.disabled = true;
+    setBanner(
+      "REPLAY MODE — this is a pre-recorded session. Run the server locally to send your own prompts.",
+      "replay-banner",
+    );
+    if (ui.speed) ui.speed.style.display = "";
+    if (ui.playPause) ui.playPause.style.display = "";
+    if (ui.restart) ui.restart.style.display = "";
+  } else {
+    ui.connection.textContent = "connecting…";
+    ui.connection.classList.add("offline");
+  }
+}
+
 function initPool(numBlocks) {
   if (state.numBlocks === numBlocks && state.poolEls.length === numBlocks) return;
   state.numBlocks = numBlocks;
@@ -68,13 +119,9 @@ function renderPool(pool) {
     const rc = pool.ref_counts[i];
     const hashed = pool.hashed[i];
     let cls = "block";
-    if (rc === 0) {
-      cls += hashed ? " cached" : " free";
-    } else if (rc === 1) {
-      cls += " used";
-    } else {
-      cls += " shared";
-    }
+    if (rc === 0) cls += hashed ? " cached" : " free";
+    else if (rc === 1) cls += " used";
+    else cls += " shared";
     if (hashed) cls += " hashed";
     el.className = cls;
     el.title = `block ${i} — refcount=${rc}${hashed ? " — hashed (cacheable)" : ""}`;
@@ -95,11 +142,10 @@ function renderPool(pool) {
 function renderSeqs(snapshot) {
   ui.schedStep.textContent = ` — step ${snapshot.step}`;
   const all = [...snapshot.running, ...snapshot.waiting];
-  // index for later token-delta merges
   state.seqsBySeqId = new Map(all.map(s => [s.seq_id, s]));
   ui.seqs.innerHTML = "";
   if (all.length === 0) {
-    ui.seqs.innerHTML = `<div class="muted">(no active sequences — send a prompt above)</div>`;
+    ui.seqs.innerHTML = `<div class="muted">(no active sequences${state.mode === 'replay' ? '' : ' — send a prompt above'})</div>`;
     return;
   }
   for (const s of all) {
@@ -129,7 +175,7 @@ function renderSeqs(snapshot) {
         </span>
       </div>
       <div class="seq-blocks">${blocksHTML || '<span class="muted">(no blocks yet)</span>'}</div>
-      <div class="seq-text"><span class="prompt">${escapeHtml(promptText)}</span><span class="gen">${escapeHtml(gen)}</span>${s.status === 'running' || s.status === 'prefilling' ? '<span class="cursor">&nbsp;</span>' : ''}</div>
+      <div class="seq-text"><span class="prompt">${escapeHtml(promptText)}</span><span class="gen">${escapeHtml(gen)}</span>${(s.status === 'running' || s.status === 'prefilling') ? '<span class="cursor">&nbsp;</span>' : ''}</div>
     `;
     ui.seqs.appendChild(div);
   }
@@ -139,6 +185,27 @@ function escapeHtml(s) {
   return (s || "").replace(/[&<>"]/g, c => ({"&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;"}[c]));
 }
 
+function applyDeltas(deltas) {
+  if (!deltas) return;
+  for (const d of deltas) {
+    let rec = state.requests.get(d.request_id);
+    if (!rec) {
+      rec = { promptText: "(prompt unknown)", generated: "", finished: false };
+      state.requests.set(d.request_id, rec);
+    }
+    if (d.new_text) rec.generated += d.new_text;
+    if (d.finished) {
+      rec.finished = true;
+      rec.finishReason = d.finish_reason;
+    }
+    const card = document.getElementById(`seq-${d.request_id}`);
+    if (card) {
+      const t = card.querySelector(".seq-text .gen");
+      if (t) t.textContent = rec.generated;
+    }
+  }
+}
+
 function handleEvent(ev) {
   if (ev.type === "snapshot") {
     const snap = ev.payload;
@@ -147,6 +214,17 @@ function handleEvent(ev) {
     renderSeqs(snap);
     return;
   }
+  if (ev.type === "request") {
+    // From the recording: capture prompt text + max_tokens for the UI.
+    const p = ev.payload;
+    state.requests.set(p.request_id, {
+      promptText: p.prompt,
+      generated: "",
+      finished: false,
+    });
+    logLine(`request ${p.request_id.slice(0,8)} — prompt=${p.prompt_len}t max_tokens=${p.max_tokens}`, "ev-admit");
+    return;
+  }
   if (ev.type === "step") {
     const p = ev.payload;
     ui.statTokens.textContent = p.num_tokens;
@@ -154,50 +232,134 @@ function handleEvent(ev) {
     ui.statMs.textContent = p.duration_ms.toFixed(1);
     if (p.preempted?.length) state.preempted += p.preempted.length;
     ui.statPre.textContent = state.preempted;
+    applyDeltas(p.deltas);
     renderPool(p.snapshot.block_pool);
     renderSeqs(p.snapshot);
 
     let msg = `step ${ev.step}: ${p.num_tokens}t (${p.num_prefill_seqs}P/${p.num_decode_seqs}D) in ${p.duration_ms.toFixed(1)}ms`;
     let cls = "ev-step";
-    if (p.newly_admitted?.length) {
-      msg += ` · admitted seq=${p.newly_admitted.join(",")}`;
-      cls = "ev-admit";
-    }
-    if (p.finished?.length) {
-      msg += ` · finished ${p.finished.map(r => r.slice(0,8)).join(",")}`;
-      cls = "ev-finish";
-    }
-    if (p.preempted?.length) {
-      msg += ` · PREEMPTED seq=${p.preempted.join(",")}`;
-      cls = "ev-preempt";
-    }
+    if (p.newly_admitted?.length) { msg += ` · admitted seq=${p.newly_admitted.join(",")}`; cls = "ev-admit"; }
+    if (p.finished?.length) { msg += ` · finished ${p.finished.map(r => r.slice(0,8)).join(",")}`; cls = "ev-finish"; }
+    if (p.preempted?.length) { msg += ` · PREEMPTED seq=${p.preempted.join(",")}`; cls = "ev-preempt"; }
     logLine(msg, cls);
   }
 }
 
-function connectEvents() {
+// ---------- live mode (SSE) ----------
+
+function connectLive() {
   const es = new EventSource("/engine/events");
-  es.onopen = () => {
-    ui.connection.textContent = "connected";
-    ui.connection.classList.remove("offline");
-    ui.connection.classList.add("online");
-  };
+  let gotOne = false;
+  es.onopen = () => { /* wait for first message to confirm live */ };
   es.onerror = () => {
-    ui.connection.textContent = "disconnected";
-    ui.connection.classList.remove("online");
-    ui.connection.classList.add("offline");
+    if (!gotOne) {
+      es.close();
+      startReplay();  // fall back
+    } else {
+      ui.connection.textContent = "disconnected";
+      ui.connection.classList.remove("online");
+      ui.connection.classList.add("offline");
+    }
   };
   es.onmessage = (e) => {
     if (!e.data) return;
-    try {
-      handleEvent(JSON.parse(e.data));
-    } catch (err) {
-      console.error("bad event", err, e.data);
-    }
+    if (!gotOne) { gotOne = true; setMode("live"); }
+    try { handleEvent(JSON.parse(e.data)); }
+    catch (err) { console.error("bad event", err, e.data); }
   };
+  // Give the server a couple seconds to respond before falling back.
+  setTimeout(() => {
+    if (!gotOne) {
+      es.close();
+      startReplay();
+    }
+  }, 2000);
 }
 
+// ---------- replay mode ----------
+
+async function startReplay() {
+  setMode("replay");
+  const params = new URLSearchParams(location.search);
+  const url = params.get("session") || "events.jsonl";
+  let text;
+  try {
+    const resp = await fetch(url, { cache: "no-cache" });
+    if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
+    text = await resp.text();
+  } catch (e) {
+    setBanner(
+      `Could not load recording (${url}). Run the server locally or commit a web/events.jsonl recording.`,
+      "replay-banner error",
+    );
+    ui.connection.textContent = "no recording";
+    return;
+  }
+  const events = text.split("\n").filter(Boolean).map(l => JSON.parse(l));
+  if (events.length === 0) {
+    setBanner("Recording is empty.", "replay-banner error");
+    return;
+  }
+  state.replay = new Replayer(events);
+  state.replay.start();
+}
+
+class Replayer {
+  constructor(events) {
+    this.events = events;
+    this.idx = 0;
+    this.speed = parseFloat($("speed")?.value || "1");
+    this.paused = false;
+    this._timeout = null;
+  }
+  reset() {
+    this.stop();
+    this.idx = 0;
+    state.requests.clear();
+    state.preempted = 0;
+    ui.log.innerHTML = "";
+  }
+  setSpeed(s) {
+    this.speed = s;
+    if (!this.paused) {
+      this.stop();
+      this._schedule();
+    }
+  }
+  pause() { this.paused = true; this.stop(); }
+  resume() { if (!this.paused) return; this.paused = false; this._schedule(); }
+  stop() {
+    if (this._timeout) clearTimeout(this._timeout);
+    this._timeout = null;
+  }
+  start() {
+    this.reset();
+    this._schedule(0);
+  }
+  _schedule(delayOverride) {
+    if (this.idx >= this.events.length) {
+      logLine("(replay complete — press Restart to replay)", "ev-finish");
+      return;
+    }
+    let delay = 0;
+    if (delayOverride !== undefined) {
+      delay = delayOverride;
+    } else if (this.idx > 0) {
+      const gap = this.events[this.idx].timestamp - this.events[this.idx - 1].timestamp;
+      delay = Math.max(0, Math.min(gap, 1.0)) * 1000 / this.speed;  // cap at 1s
+    }
+    this._timeout = setTimeout(() => {
+      const ev = this.events[this.idx++];
+      try { handleEvent(ev); } catch (e) { console.error(e); }
+      if (!this.paused) this._schedule();
+    }, delay);
+  }
+}
+
+// ---------- live: prompt submission ----------
+
 async function sendPrompt(prompt) {
+  if (state.mode !== "live") return;
   const body = {
     prompt,
     max_tokens: parseInt($("max_tokens").value, 10),
@@ -215,8 +377,6 @@ async function sendPrompt(prompt) {
     logLine(`request failed: ${txt}`, "ev-preempt");
     return;
   }
-
-  // Parse SSE manually so we can read each event as it arrives.
   const reader = resp.body.getReader();
   const decoder = new TextDecoder();
   let buf = "";
@@ -242,31 +402,43 @@ async function sendPrompt(prompt) {
         if (j.text) rec.generated += j.text;
         rec.finished = j.finished;
         rec.finishReason = j.finish_reason;
-        // Repaint the matching seq card if visible.
         const card = document.getElementById(`seq-${myReqId}`);
         if (card) {
-          const text = card.querySelector(".seq-text .gen");
-          if (text) text.textContent = rec.generated;
+          const t = card.querySelector(".seq-text .gen");
+          if (t) t.textContent = rec.generated;
         }
-      } catch (e) {
-        console.error("bad chunk", e, data);
-      }
+      } catch (e) { console.error("bad chunk", e, data); }
     }
   }
 }
 
-ui.send.addEventListener("click", () => sendPrompt($("prompt").value));
+ui.send.addEventListener("click", () => sendPrompt(ui.prompt.value));
 ui.sendTwice.addEventListener("click", async () => {
-  const p = $("prompt").value;
-  // First send fills the prefix cache; second send should hit it.
+  const p = ui.prompt.value;
   await sendPrompt(p);
   await new Promise(r => setTimeout(r, 200));
   await sendPrompt(p);
 });
-$("prompt").addEventListener("keydown", (e) => {
-  if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
-    sendPrompt(e.target.value);
-  }
+ui.prompt.addEventListener("keydown", (e) => {
+  if ((e.metaKey || e.ctrlKey) && e.key === "Enter") sendPrompt(e.target.value);
 });
 
-connectEvents();
+if (ui.speed) ui.speed.addEventListener("change", () => {
+  state.replay?.setSpeed(parseFloat(ui.speed.value));
+});
+if (ui.playPause) ui.playPause.addEventListener("click", () => {
+  if (!state.replay) return;
+  if (state.replay.paused) { state.replay.resume(); ui.playPause.textContent = "Pause"; }
+  else { state.replay.pause(); ui.playPause.textContent = "Play"; }
+});
+if (ui.restart) ui.restart.addEventListener("click", () => state.replay?.start());
+
+// ---------- entry point ----------
+
+(function boot() {
+  setMode("connecting");
+  const force = new URLSearchParams(location.search).get("mode");
+  if (force === "replay") startReplay();
+  else if (force === "live") connectLive();
+  else connectLive();   // will auto-fall-back to replay on no-response
+})();

web/events.jsonl

@@ -0,0 +1,21 @@
+{"type":"snapshot","step":0,"timestamp":0.0,"payload":{"step":0,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":24,"num_cached_entries":0,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[],"waiting":[]}}
+{"type":"request","step":1,"timestamp":0.5,"payload":{"request_id":"demo-aaaa1111","seq_id":1,"prompt":"Explain paged attention in two sentences. Then explain prefix caching.","prompt_len":20,"max_tokens":24}}
+{"type":"step","step":1,"timestamp":0.55,"payload":{"duration_ms":280,"num_tokens":16,"num_seqs":1,"num_prefill_seqs":1,"num_decode_seqs":0,"deltas":[],"newly_admitted":[1],"finished":[],"preempted":[],"snapshot":{"step":1,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":22,"num_cached_entries":2,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[],"waiting":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"prefilling","prompt_len":20,"num_generated":0,"num_computed_tokens":16,"num_cached_prefix_tokens":0,"block_table":[0,1]}]}}}
+{"type":"step","step":2,"timestamp":0.9,"payload":{"duration_ms":180,"num_tokens":4,"num_seqs":1,"num_prefill_seqs":1,"num_decode_seqs":0,"deltas":[],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":2,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":21,"num_cached_entries":2,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":0,"num_computed_tokens":20,"num_cached_prefix_tokens":0,"block_table":[0,1,2]}],"waiting":[]}}}
+{"type":"step","step":3,"timestamp":1.22,"payload":{"duration_ms":95,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" Paged","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":3,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":21,"num_cached_entries":2,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":1,"num_computed_tokens":21,"num_cached_prefix_tokens":0,"block_table":[0,1,2]}],"waiting":[]}}}
+{"type":"step","step":4,"timestamp":1.34,"payload":{"duration_ms":95,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" attention","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":4,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":21,"num_cached_entries":2,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":2,"num_computed_tokens":22,"num_cached_prefix_tokens":0,"block_table":[0,1,2]}],"waiting":[]}}}
+{"type":"step","step":5,"timestamp":1.46,"payload":{"duration_ms":95,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" splits","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":5,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":21,"num_cached_entries":2,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":3,"num_computed_tokens":23,"num_cached_prefix_tokens":0,"block_table":[0,1,2]}],"waiting":[]}}}
+{"type":"step","step":6,"timestamp":1.58,"payload":{"duration_ms":95,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" keys","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":6,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":21,"num_cached_entries":3,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":4,"num_computed_tokens":24,"num_cached_prefix_tokens":0,"block_table":[0,1,2]}],"waiting":[]}}}
+{"type":"step","step":7,"timestamp":1.7,"payload":{"duration_ms":95,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" and","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":7,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":20,"num_cached_entries":3,"prefix_cache_hits":0,"prefix_cache_lookups":0,"ref_counts":[1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":5,"num_computed_tokens":25,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]}],"waiting":[]}}}
+{"type":"request","step":8,"timestamp":2.3,"payload":{"request_id":"demo-bbbb2222","seq_id":2,"prompt":"Explain paged attention in two sentences. Then explain prefix caching.","prompt_len":20,"max_tokens":24}}
+{"type":"step","step":9,"timestamp":2.35,"payload":{"duration_ms":140,"num_tokens":5,"num_seqs":2,"num_prefill_seqs":1,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" values","finished":false,"finish_reason":null}],"newly_admitted":[2],"finished":[],"preempted":[],"snapshot":{"step":9,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":19,"num_cached_entries":3,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":5,"num_computed_tokens":25,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]}],"waiting":[{"seq_id":2,"request_id":"demo-bbbb2222","status":"prefilling","prompt_len":20,"num_generated":0,"num_computed_tokens":16,"num_cached_prefix_tokens":16,"block_table":[0,1,4]}]}}}
+{"type":"step","step":10,"timestamp":2.47,"payload":{"duration_ms":110,"num_tokens":5,"num_seqs":2,"num_prefill_seqs":1,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" into","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":10,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":19,"num_cached_entries":3,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":6,"num_computed_tokens":26,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":0,"num_computed_tokens":20,"num_cached_prefix_tokens":16,"block_table":[0,1,4]}],"waiting":[]}}}
+{"type":"step","step":11,"timestamp":2.69,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":" small","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" Prefix","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":11,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":19,"num_cached_entries":3,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":7,"num_computed_tokens":27,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":1,"num_computed_tokens":21,"num_cached_prefix_tokens":16,"block_table":[0,1,4]}],"waiting":[]}}}
+{"type":"step","step":12,"timestamp":2.79,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":",","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" caching","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":12,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":19,"num_cached_entries":3,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":8,"num_computed_tokens":28,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":2,"num_computed_tokens":22,"num_cached_prefix_tokens":16,"block_table":[0,1,4]}],"waiting":[]}}}
+{"type":"step","step":13,"timestamp":2.89,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":" fixed","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" reuses","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":13,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":19,"num_cached_entries":3,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":9,"num_computed_tokens":29,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":3,"num_computed_tokens":23,"num_cached_prefix_tokens":16,"block_table":[0,1,4]}],"waiting":[]}}}
+{"type":"step","step":14,"timestamp":2.99,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":"-size","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" those","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":14,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":19,"num_cached_entries":4,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":10,"num_computed_tokens":30,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":4,"num_computed_tokens":24,"num_cached_prefix_tokens":16,"block_table":[0,1,4]}],"waiting":[]}}}
+{"type":"step","step":15,"timestamp":3.09,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":" blocks","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" blocks","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":15,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":18,"num_cached_entries":4,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":11,"num_computed_tokens":31,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":5,"num_computed_tokens":25,"num_cached_prefix_tokens":16,"block_table":[0,1,4,5]}],"waiting":[]}}}
+{"type":"step","step":16,"timestamp":3.19,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":" stored","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" across","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":16,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":18,"num_cached_entries":4,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":12,"num_computed_tokens":32,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":6,"num_computed_tokens":26,"num_cached_prefix_tokens":16,"block_table":[0,1,4,5]}],"waiting":[]}}}
+{"type":"step","step":17,"timestamp":3.29,"payload":{"duration_ms":105,"num_tokens":2,"num_seqs":2,"num_prefill_seqs":0,"num_decode_seqs":2,"deltas":[{"request_id":"demo-aaaa1111","new_text":" in","finished":false,"finish_reason":null},{"request_id":"demo-bbbb2222","new_text":" requests","finished":false,"finish_reason":null}],"newly_admitted":[],"finished":[],"preempted":[],"snapshot":{"step":17,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":18,"num_cached_entries":4,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[2,2,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,false,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":1,"request_id":"demo-aaaa1111","status":"running","prompt_len":20,"num_generated":13,"num_computed_tokens":33,"num_cached_prefix_tokens":0,"block_table":[0,1,2,3]},{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":7,"num_computed_tokens":27,"num_cached_prefix_tokens":16,"block_table":[0,1,4,5]}],"waiting":[]}}}
+{"type":"step","step":18,"timestamp":3.39,"payload":{"duration_ms":85,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-aaaa1111","new_text":" the GPU.","finished":true,"finish_reason":"stop"}],"newly_admitted":[],"finished":["demo-aaaa1111"],"preempted":[],"snapshot":{"step":18,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":20,"num_cached_entries":5,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[1,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[{"seq_id":2,"request_id":"demo-bbbb2222","status":"running","prompt_len":20,"num_generated":8,"num_computed_tokens":28,"num_cached_prefix_tokens":16,"block_table":[0,1,4,5]}],"waiting":[]}}}
+{"type":"step","step":19,"timestamp":3.49,"payload":{"duration_ms":80,"num_tokens":1,"num_seqs":1,"num_prefill_seqs":0,"num_decode_seqs":1,"deltas":[{"request_id":"demo-bbbb2222","new_text":" for the same prompts.","finished":true,"finish_reason":"stop"}],"newly_admitted":[],"finished":["demo-bbbb2222"],"preempted":[],"snapshot":{"step":19,"config":{"model":"Qwen/Qwen2.5-0.5B-Instruct","block_size":8,"num_blocks":24,"max_num_seqs":8,"max_num_batched_tokens":32,"prefix_caching":true},"block_pool":{"num_blocks":24,"block_size":8,"num_free_blocks":24,"num_cached_entries":6,"prefix_cache_hits":2,"prefix_cache_lookups":2,"ref_counts":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"hashed":[true,true,true,true,true,true,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false,false]},"running":[],"waiting":[]}}}

web/index.html

@@ -10,11 +10,13 @@
 <header>
   <h1>tiny_vllm <span class="muted">— minimal continuous-batching engine</span></h1>
   <div class="status">
-    <span id="connection" class="badge offline">disconnected</span>
+    <span id="connection" class="badge offline">connecting…</span>
     <span id="model" class="muted"></span>
   </div>
 </header>
 
+<div id="banner" class="banner" style="display:none"></div>
+
 <section class="prompt-box">
   <textarea id="prompt" rows="2" placeholder="Type a prompt and press Send (or Cmd/Ctrl+Enter)…">Explain paged attention in two sentences.</textarea>
   <div class="controls">
@@ -23,6 +25,18 @@
     <label>top_p <input id="top_p" type="number" value="0.9" step="0.05" min="0" max="1"></label>
     <button id="send">Send</button>
     <button id="send-twice" title="Submit the same prompt twice — second should hit prefix cache">Send ×2 (prefix demo)</button>
+
+    <span class="replay-controls">
+      <select id="speed" style="display:none" title="Replay speed">
+        <option value="0.5">0.5×</option>
+        <option value="1" selected>1×</option>
+        <option value="2">2×</option>
+        <option value="4">4×</option>
+        <option value="8">8×</option>
+      </select>
+      <button id="play-pause" style="display:none" class="ghost">Pause</button>
+      <button id="restart" style="display:none" class="ghost">Restart</button>
+    </span>
   </div>
 </section>
 

web/style.css

@@ -40,6 +40,17 @@ header h1 { font-size: 16px; margin: 0; font-weight: 600; }
 }
 .badge.online { background: rgba(63, 185, 80, 0.15); color: var(--green); }
 .badge.offline { background: rgba(248, 81, 73, 0.15); color: var(--red); }
+.badge.replay { background: rgba(163, 113, 247, 0.15); color: var(--purple); }
+
+.banner {
+  padding: 8px 20px;
+  font-size: 12px;
+  background: rgba(163, 113, 247, 0.12);
+  color: var(--purple);
+  border-bottom: 1px solid rgba(163, 113, 247, 0.3);
+}
+.banner.error { background: rgba(248, 81, 73, 0.12); color: var(--red); border-bottom-color: rgba(248, 81, 73, 0.3); }
+.banner.replay-banner.error { background: rgba(248, 81, 73, 0.12); color: var(--red); }
 
 .prompt-box {
   padding: 12px 20px;
@@ -70,8 +81,22 @@ button {
   padding: 6px 14px; font-weight: 500; cursor: pointer;
 }
 button:hover { filter: brightness(1.1); }
+button:disabled { opacity: 0.4; cursor: not-allowed; }
+button.ghost {
+  background: transparent;
+  border: 1px solid var(--border);
+  color: var(--fg);
+}
 #send-twice { background: var(--purple); }
 
+textarea:disabled { opacity: 0.5; }
+
+.replay-controls { margin-left: auto; display: flex; gap: 6px; align-items: center; }
+.replay-controls select {
+  background: var(--bg); color: var(--fg); border: 1px solid var(--border);
+  border-radius: 4px; padding: 4px 6px; font-family: var(--mono);
+}
+
 main {
   display: grid;
   grid-template-columns: 1fr 1fr;