Knowledge Lifecycle

index.html

@@ -29,8 +29,8 @@
 <header>
   <h1>LLM Knowledge Lifecycle</h1>
   <p class="subtitle">
-    A language model carries two memories: what it learned in training, and what you place in
-    its prompt. When they disagree, which one wins? Companion demonstration for
+    A model holds two memories: what it learned in training, and what you put in its prompt.
+    When they disagree, which one wins? Companion demonstration for
     <em>The Knowledge Lifecycle of Large Language Models</em>.
   </p>
   <p class="badges">
@@ -41,8 +41,8 @@
   </p>
   <div class="alert a-important">
     <span class="alert-title">Important</span>
-    The model runs entirely in your browser. Every measurement is deterministic and computed
-    locally; nothing leaves your machine. First measurement downloads the model once (128 MB).
+    The model runs in your browser. Nothing leaves your machine, and identical inputs always
+    give identical numbers. The first measurement downloads the model once, 128 MB.
   </div>
 </header>
 
@@ -62,10 +62,8 @@
 <section id="probe" class="panel active">
 
   <p class="lede">
-    Each preset is a real, documented fact change that happened after this model's training
-    data was collected. The model's weights hold the old world; the corrective document in the
-    prompt holds the new one. Pick a case, press Measure, and watch which memory controls the
-    answer.
+    Each preset is a documented fact change from after this model was trained. Its weights hold
+    the old world, the document in the prompt holds the new one. Press Measure to see which wins.
   </p>
 
   <div class="presets">
@@ -102,7 +100,7 @@
     <p id="load-status" class="dim"></p>
   </div>
 
-  <div id="results" hidden>
+  <div id="results" role="region" aria-live="polite" aria-label="Measurement results" hidden>
 
     <div class="verdict-row">
       <span id="verdict-chip" class="chip"></span>
@@ -112,13 +110,13 @@
     <p id="narrative" class="narrative"></p>
 
     <h3>The two distributions, side by side</h3>
-    <p class="dim small">Next-token probability without the document (grey) and with it (blue).
-    The correct answer is outlined in green; if its bar is invisible, that is the finding.</p>
+    <p class="dim small">Probability of each candidate without the document (grey) and with it (blue).
+    The correct answer is outlined in green. If its bar is invisible, that is the finding.</p>
     <div id="bars"></div>
 
     <div class="gauge-block">
       <div class="gauge-label">
-        <span>D<sub>sync</sub> &nbsp;<span class="dim">how lost the correct answer is</span></span>
+        <span>D<sub>sync</sub> &nbsp;<span class="dim">how lost the correct answer is, in nats</span></span>
         <strong id="r-dsync"></strong>
       </div>
       <div class="gauge">
@@ -130,7 +128,7 @@
 
     <div class="gauge-block">
       <div class="gauge-label">
-        <span>I<sub>ctx</sub> &nbsp;<span class="dim">how much the document moved the model</span></span>
+        <span>I<sub>ctx</sub> &nbsp;<span class="dim">how much the document moved the model, in nats</span></span>
         <strong id="r-ictx"></strong>
       </div>
       <div class="gauge slim">
@@ -142,7 +140,7 @@
 
     <div class="temp-block">
       <div class="gauge-label">
-        <span>Sampling temperature <span class="dim">(recomputed live from the measured logits)</span></span>
+        <span>Sampling temperature <span class="dim">(how randomly the model picks its answer)</span></span>
         <strong id="t-value">1.0</strong>
       </div>
       <input id="temp" type="range" min="0.2" max="2.0" step="0.1" value="1.0">
@@ -155,8 +153,8 @@
         <tr><td>P(answer | query alone)</td><td id="r-p0"></td></tr>
         <tr><td>P(answer | context + query)</td><td id="r-p1"></td></tr>
         <tr><td>Answer tokenization</td><td id="r-tok"></td></tr>
-        <tr><td>D<sub>sync</sub> definition</td><td>&minus;ln P(answer | context + query), in nats</td></tr>
-        <tr><td>I<sub>ctx</sub> definition</td><td>KL(with context &Vert; without), full vocabulary</td></tr>
+        <tr><td>D<sub>sync</sub> definition</td><td>&minus;ln P(answer given context and query): the negative natural logarithm of the correct answer's probability</td></tr>
+        <tr><td>I<sub>ctx</sub> definition</td><td>Kullback-Leibler divergence between the with-document and without-document distributions, over all 50,257 tokens</td></tr>
       </table>
     </details>
   </div>
@@ -194,6 +192,19 @@
     <tr><td><span class="chip chip-influence">drift</span></td><td><strong>Twitter rename.</strong> Context lifts the right answer by orders of magnitude and still loses to " Twitter".</td></tr>
   </table>
 
+
+  <h2>Terms used on this page</h2>
+  <table class="glossary">
+    <tr><td><strong>Token</strong></td><td>The unit a model reads and writes. Roughly a word or word fragment; " withdrawn" is one token, and the leading space is part of it.</td></tr>
+    <tr><td><strong>Parametric memory</strong></td><td>What the model absorbed into its weights during training. Fixed after training, and the model cannot tell you when it learned any of it.</td></tr>
+    <tr><td><strong>Context</strong></td><td>What you put in the prompt right now, including any document retrieved for the model to read.</td></tr>
+    <tr><td><strong>Nat</strong></td><td>A unit of information, measured with natural logarithms. Here it converts directly to probability: a value of <em>n</em> nats means the correct answer holds probability e<sup>&minus;n</sup>. One nat is roughly 37%, nine nats is roughly one in ten thousand.</td></tr>
+    <tr><td><strong>Surprisal</strong></td><td>How surprised the model is by an answer, written as &minus;ln P. Low when the model expected it, high when it did not. D<sub>sync</sub> is the surprisal of the correct answer.</td></tr>
+    <tr><td><strong>KL divergence</strong></td><td>A measure of how far one probability distribution sits from another. I<sub>ctx</sub> uses it to ask whether the document changed the model's mind about anything at all.</td></tr>
+    <tr><td><strong>Temperature</strong></td><td>How randomly a model picks among candidate answers. Low values make it repeat its favorite; high values spread the choice out.</td></tr>
+    <tr><td><strong>Quantized</strong></td><td>Weights stored at reduced precision so the model downloads and runs faster. This page uses 8-bit weights, which shift individual probabilities slightly without changing any conclusion.</td></tr>
+  </table>
+
   <h2>Build your own probe</h2>
   <p>
     Any fact change with a single-word answer works. Three rules make a clean probe: the query

main.js

@@ -1,9 +1,11 @@
 // =============================================================================
 // File        : main.js
 // Project     : The Knowledge Lifecycle of Large Language Models
-// Purpose     : Browser-side D_sync measurement engine: loads GPT-2, runs both prompt
-// conditions, and renders the narrative, bars, gauges, and diagnostics.
-// Tech Stack  : JavaScript (ES modules), transformers.js 3.4.0, ONNX Runtime Web (WASM)
+// Purpose     : Browser-side measurement engine. Loads GPT-2, runs the two
+//               prompt conditions, and renders the narrative, the paired
+//               distributions, the gauges, and the stage diagnosis.
+// Tech Stack  : JavaScript (ES modules), transformers.js 3.4.0,
+//               ONNX Runtime Web (WASM)
 // Authors     : Amey Thakur (https://github.com/Amey-Thakur)
 //               Sarvesh Talele (https://github.com/sarveshtalele)
 // Repository  : https://github.com/Amey-Thakur/LLM-KNOWLEDGE-LIFECYCLE
@@ -23,19 +25,19 @@ const PRESETS = {
     query: "Question: Is Vioxx safe to prescribe? Answer: Vioxx is considered",
     context: "Context: In September 2004, Merck voluntarily withdrew Vioxx after trials revealed increased cardiovascular risks.",
     answer: " withdrawn",
-    reading: "The paper's headline case. The withdrawal notice sits in the prompt, and the model still answers “safe”. Expect D_sync deep past the 9.2-nat failure threshold; the paper's fp32 run measures 12.05 nats with I_ctx = 0.033.",
+    reading: "The headline case. The withdrawal notice is in the prompt and the model still answers “safe”. At full precision the paper measures 12.05 nats, far past the 9.2 failure threshold.",
   },
   monarch: {
     query: "Question: Who is the current British monarch? Answer: The current British monarch is",
     context: "Context: Queen Elizabeth II died in September 2022. Charles III acceded to the throne and is the reigning King of the United Kingdom.",
     answer: " Charles",
-    reading: "A subtler failure. The context raises P(Charles), but its strongest effect is boosting “ Queen”: merely mentioning the late monarch reinforces the stale association. Correct information can strengthen the wrong answer.",
+    reading: "A subtler failure. The document does raise the correct answer, but its strongest effect is boosting “ Queen”. Naming a fact, even to correct it, reinforces the old association.",
   },
   twitter: {
     query: "Question: What is the social network Twitter called today? Answer: Twitter is now called",
     context: "Context: In July 2023, Twitter was rebranded as X under Elon Musk's ownership.",
     answer: " X",
-    reading: "The context moves the distribution hard (the paper's fp32 run: I_ctx = 0.9 nats) and lifts the correct answer by orders of magnitude. The model still answers “Twitter”. Influence without resolution.",
+    reading: "Here the document moves the model hard and lifts the correct answer by orders of magnitude. It still answers “Twitter”. Influence without resolution.",
   },
 };
 
@@ -115,18 +117,24 @@ function softmax(row, temperature = 1.0) {
   return probs;
 }
 
+/**
+ * Indices of the k largest probabilities, highest first.
+ *
+ * One pass over the vocabulary, maintaining a short sorted list. The obvious
+ * alternative, k full scans with a "already taken" set, costs k passes over
+ * 50,257 entries at every call site; this costs one.
+ */
 function topK(probs, k) {
-  const taken = new Set();
-  const idx = [];
-  for (let n = 0; n < k; n++) {
-    let best = -1, bp = -1;
-    for (let i = 0; i < probs.length; i++) {
-      if (!taken.has(i) && probs[i] > bp) { bp = probs[i]; best = i; }
-    }
-    taken.add(best);
-    idx.push(best);
+  const best = [];   // indices, kept sorted by descending probability
+  for (let i = 0; i < probs.length; i++) {
+    const p = probs[i];
+    if (best.length === k && p <= probs[best[k - 1]]) continue;
+    let at = best.length;
+    while (at > 0 && probs[best[at - 1]] < p) at--;
+    best.splice(at, 0, i);
+    if (best.length > k) best.pop();
   }
-  return idx;
+  return best;
 }
 
 // ---------- rendering ----------
@@ -158,24 +166,35 @@ function renderBars(pPlain, pCtx, target) {
     + rows.join("");
 }
 
+/**
+ * Plain-language account of one measurement, written from the numbers.
+ *
+ * The observations are additive rather than exclusive: a single run can be
+ * both "the document helped" and "the document also strengthened the wrong
+ * answer", and an if/else chain would report only the first and hide the
+ * second, which is the more interesting half.
+ */
 function narrative(m) {
-  const s = [];
-  const topCtxTok = fmtTok(m.topCtx);
-  const ansTok = fmtTok(m.answerPiece).trim();
-  s.push(`With the corrective document in its prompt, the model's most likely continuation is “${topCtxTok}” at ${fmtPct(m.pTopCtx)}.`);
+  const answerWon = m.topCtx === m.answerPiece;
   const odds = Math.round(1 / m.pt1);
-  s.push(`The correct answer “${ansTok}” receives ${fmtPct(m.pt1)}: about one chance in ${odds.toLocaleString()}.`);
-  if (m.topCtx === m.topPlain && m.ictx < 0.05) {
-    s.push(`The document changed almost nothing. The model's whole distribution moved by ${m.ictx.toFixed(3)} nats, and its preferred answer is the same one it gives with no document at all.`);
-  } else if (m.pt1 > m.pt0 * 1.5 && m.topCtx !== fmtTok(m.answerPiece)) {
-    const factor = (m.pt1 / m.pt0).toFixed(1);
-    s.push(`The document helped: it multiplied the correct answer's probability by ${factor}. It still loses.`);
-  } else if (m.pTopCtx > m.pTopPlainSameTok) {
-    s.push(`Counterintuitively, the document made the leading wrong answer stronger, raising it from ${fmtPct(m.pTopPlainSameTok)} to ${fmtPct(m.pTopCtx)}. Mentioning a fact, even to correct it, reinforces the association.`);
+  const s = [
+    `With the corrective document in its prompt, the model's most likely continuation is “${m.topCtx}” at ${fmtPct(m.pTopCtx)}.`,
+    `The correct answer “${m.answerPiece.trim()}” receives ${fmtPct(m.pt1)}: about one chance in ${odds.toLocaleString()}.`,
+  ];
+
+  if (m.ictx < 0.05) {
+    s.push(`The document barely registered: it moved the model's whole distribution by ${m.ictx.toFixed(3)} nats.`);
   }
-  if (m.topCtx === fmtTok(m.answerPiece)) {
-    s.push("Here the document won: the model's top answer is the correct one. This is what synchronization looks like.");
+  if (m.pt1 > m.pt0 * 1.5) {
+    s.push(`It did help the correct answer, multiplying its probability by ${(m.pt1 / m.pt0).toFixed(1)}.`);
   }
+  if (!answerWon && m.pTopCtx > m.pTopPlainSameTok) {
+    s.push(`It also strengthened the leading wrong answer, raising “${m.topCtx}” from ${fmtPct(m.pTopPlainSameTok)} to ${fmtPct(m.pTopCtx)}: mentioning a fact, even to correct it, reinforces the association.`);
+  }
+
+  s.push(answerWon
+    ? "The document won here: the model's top answer is the correct one. This is what synchronization looks like."
+    : "The parametric memory still controls the answer.");
   return s.join(" ");
 }
 
@@ -286,7 +305,7 @@ $("run").addEventListener("click", async () => {
 
     const m = {
       pt0, pt1, dsync, ictx,
-      answerPiece: pieces[0],
+      answerPiece: fmtTok(pieces[0]),
       topPlain: fmtTok(tokenizer.decode([topPlainId])),
       topCtx: fmtTok(tokenizer.decode([topCtxId])),
       pTopCtx: pCtx[topCtxId],

style.css

@@ -276,3 +276,69 @@ textarea:focus, input:focus, select:focus, #run:focus-visible, .preset:focus-vis
   outline: 2px solid var(--accent); outline-offset: 1px;
 }
 .stagebar { border-radius: 6px; overflow: hidden; }
+
+/* narrow screens: stack the paired bars rather than crushing four columns */
+@media (max-width: 620px) {
+  body { padding: 2rem 1rem 3rem; }
+  .bar-tok { flex: 0 0 5rem; font-size: .78rem; }
+  .bar-pair { grid-template-columns: 1fr 4rem; row-gap: .2rem; }
+  .row { flex-wrap: wrap; }
+  #run { width: 100%; }
+  .authors { gap: 1.5rem; }
+  .stagebar { font-size: .62rem; }
+  .gauge-ticks span { font-size: .66rem; }
+}
+
+/* ---------------------------------------------------------------------------
+   Typographic measure and rhythm.
+
+   Prose is capped near 68 characters. Beyond that the eye loses the line
+   start on the return sweep, which is what made the earlier full-width
+   paragraphs tiring to read. Tables, bars, and gauges stay full width: they
+   are scanned, not read.
+   --------------------------------------------------------------------------- */
+
+body { line-height: 1.65; }
+
+.subtitle { max-width: 38rem; font-size: 1.02rem; }
+.lede,
+.reading,
+#guide p,
+.narrative,
+.note { max-width: 46rem; }
+.alert { max-width: 46rem; line-height: 1.6; }
+.subtitle { max-width: 44rem; }
+
+.lede { margin-bottom: 1.6rem; font-size: 1rem; }
+.reading { margin: 1rem 0 1.8rem; }
+
+header { margin-bottom: 2.2rem; }
+header h1 { font-size: 1.9rem; margin-bottom: .6rem; }
+.badges { margin-bottom: 1.2rem; }
+
+.panel { padding-top: 2.2rem; }
+.inputs { margin-top: 1.8rem; }
+.inputs label { margin-bottom: 1.1rem; }
+#results { margin-top: 2.8rem; }
+.verdict-row { margin-bottom: 1.4rem; }
+.narrative { padding: 1rem 1.2rem; margin-bottom: 2rem; line-height: 1.7; }
+#bars { padding: 1rem 1.2rem; margin-bottom: 2.2rem; }
+.gauge-block { margin-bottom: 2rem; }
+.temp-block { padding: 1rem 1.2rem; margin-bottom: 1.8rem; }
+
+#guide h2 { margin: 2.4rem 0 .8rem; }
+#guide h2:first-child { margin-top: .4rem; }
+#guide p { margin-bottom: 1.2rem; }
+#guide table { margin-bottom: 1.8rem; }
+th, td { padding: .55rem .7rem; }
+
+h3 { margin: 1.8rem 0 .3rem; }
+.cols h3 { margin-top: 0; }
+
+footer { margin-top: 4rem; padding-top: 2rem; }
+.authors { margin-bottom: 1.6rem; }
+
+/* glossary: term column narrow and steady so definitions align */
+.glossary td:first-child { width: 11rem; color: var(--ink); }
+.glossary td:last-child { color: var(--muted); }
+.glossary { max-width: 46rem; }