File size: 15,634 Bytes
6fd05fe fdd8183 01bb8f0 fdd8183 6fd05fe fdd8183 6fd05fe 0d15004 adcc863 6fd05fe 73308ed 10a5463 c527f4a 6fd05fe 0d15004 6fd05fe fdd8183 6fd05fe fdd8183 6fd05fe fdd8183 0d15004 fdd8183 6fd05fe 34a397a fdd8183 6fd05fe fdd8183 6fd05fe 0d15004 fdd8183 6fd05fe fdd8183 0d15004 fdd8183 6fd05fe 0d15004 6fd05fe 9ab1767 6fd05fe 0d15004 6fd05fe fdd8183 0d15004 fdd8183 6fd05fe fdd8183 0d15004 fdd8183 6fd05fe 0d15004 fdd8183 6fd05fe 01bb8f0 6fd05fe c527f4a 6fd05fe 108d191 6fd05fe c527f4a 6fd05fe fd72eef | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 | <!DOCTYPE html>
<!-- ==========================================================================
File : index.html
Project : The Knowledge Lifecycle of Large Language Models
Purpose : Page structure for the live demonstration: probe, guide,
results, and author attribution.
Authors : Amey Thakur (https://github.com/Amey-Thakur)
Sarvesh Talele (https://github.com/sarveshtalele)
Repository : https://github.com/Amey-Thakur/LLM-KNOWLEDGE-LIFECYCLE
Release Date: August 18, 2026
License : CC BY 4.0
=========================================================================== -->
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>LLM Knowledge Lifecycle</title>
<meta name="description" content="When a language model's training memory contradicts the document in its prompt, which one wins? Measure it live on GPT-2, in your browser.">
<meta property="og:title" content="The Knowledge Lifecycle of Large Language Models">
<meta property="og:description" content="GPT-2, asked about Vioxx with the 2004 withdrawal notice in its prompt, still answers safe. Measure the failure live in your browser.">
<meta property="og:type" content="website">
<meta property="og:image" content="https://huggingface.co/spaces/ameythakur/llm-knowledge-lifecycle/resolve/main/social-preview.png">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="https://huggingface.co/spaces/ameythakur/llm-knowledge-lifecycle/resolve/main/social-preview.png">
<link rel="stylesheet" href="style.css">
</head>
<body>
<header>
<h1>LLM Knowledge Lifecycle</h1>
<p class="subtitle">
A model holds two memories: what it learned in training, and what you put in its prompt.
When they disagree, which one wins?
</p>
<p class="paper-line">
Companion demonstration for <em>The Knowledge Lifecycle of Large Language Models</em>
</p>
<p class="badges">
<a href="https://github.com/Amey-Thakur/LLM-KNOWLEDGE-LIFECYCLE"><img src="https://img.shields.io/badge/GitHub-LLM--KNOWLEDGE--LIFECYCLE-181717?logo=github" alt="GitHub repository"></a>
<a href="https://www.kaggle.com/code/ameythakur20/cross-model-lifecycle-desynchronization"><img src="https://img.shields.io/badge/Notebook-Kaggle-20BEFF?logo=kaggle&logoColor=white" alt="Kaggle notebook"></a>
<a href="https://github.com/Amey-Thakur"><img src="https://img.shields.io/badge/Authors-Amey_Thakur_%26_Sarvesh_Talele-0969DA" alt="Authors"></a>
<img src="https://img.shields.io/badge/Model-GPT--2_base_(124M)-8250DF" alt="Model">
<img src="https://img.shields.io/badge/License-CC_BY_4.0-lightgrey" alt="License">
</p>
<div class="alert a-important">
<span class="alert-title">Important</span>
The model runs in your browser. Nothing leaves your machine, and identical inputs always
give identical numbers. The first measurement downloads the model once, 128 MB.
</div>
</header>
<div class="stagebar" id="stagebar" title="The five lifecycle stages">
<span class="s1">Acquire</span><span class="s2">Store</span><span class="s3" id="sb-retrieve">Retrieve</span><span class="s4" id="sb-update">Update</span><span class="s5">Forget</span>
</div>
<p class="stagenote" id="stagenote">Knowledge flows through five stages. This probe tests the boundary between Retrieve and Update.</p>
<nav class="tabs" role="tablist">
<button class="tab active" data-tab="probe" role="tab">Probe</button>
<button class="tab" data-tab="guide" role="tab">Guide</button>
</nav>
<main>
<!-- ================= PROBE ================= -->
<section id="probe" class="panel active">
<p class="lede">
Each preset is a documented fact change from after this model was trained. Its weights hold
the old world, the document in the prompt holds the new one. Press Measure to see which wins.
</p>
<div class="presets">
<button class="preset r-fail active" data-preset="vioxx">
<strong>Vioxx withdrawal</strong> 2004<span>resolution failure</span></button>
<button class="preset r-drift" data-preset="monarch">
<strong>British monarch</strong> 2022<span>the exposure trap</span></button>
<button class="preset r-influence" data-preset="twitter">
<strong>Twitter rename</strong> 2023<span>drift under strong context</span></button>
</div>
<p id="preset-reading" class="reading"></p>
<div class="inputs">
<label>Query <span class="dim">(ends mid-sentence; the next token is the answer)</span>
<textarea id="query" rows="2"></textarea>
</label>
<label>Corrective context <span class="dim">(contradicts the model's memory)</span>
<textarea id="context" rows="2"></textarea>
</label>
<div class="row">
<label class="grow">Correct continuation <span class="dim">(one word)</span>
<input id="answer" type="text">
</label>
<label>Repeat document <span class="dim">(amplify)</span>
<select id="repeat" title="How many times the corrective document is repeated in the prompt">
<option value="1" selected>1x</option>
<option value="2">2x</option>
<option value="3">3x</option>
</select>
</label>
<button id="run">Measure</button>
</div>
<div id="load-bar-wrap" hidden><div id="load-bar"></div></div>
<p id="load-status" class="dim"></p>
</div>
<div id="results" role="region" aria-live="polite" aria-label="Measurement results" hidden>
<div class="verdict-row">
<span id="verdict-chip" class="chip"></span>
<span id="r-verdict-detail" class="dim"></span>
</div>
<p id="narrative" class="narrative"></p>
<h3>The two distributions, side by side</h3>
<p class="dim small">Probability of each candidate without the document (grey) and with it (blue).
The correct answer is outlined in green. If its bar is invisible, that is the finding.</p>
<div id="bars"></div>
<div class="gauge-block">
<div class="gauge-label">
<span>D<sub>sync</sub> <span class="dim">how lost the correct answer is, in nats</span></span>
<strong id="r-dsync"></strong>
</div>
<div class="gauge">
<div class="zone z-ok" style="width:5%"></div><div class="zone z-drift" style="width:28%"></div><div class="zone z-severe" style="width:33%"></div><div class="zone z-fail" style="width:34%"></div>
<div id="g-marker" class="marker"></div>
</div>
<div class="gauge-ticks"><span style="left:5%">0.69<br><i>50%</i></span><span style="left:33%">4.6<br><i>1%</i></span><span style="left:66%">9.2<br><i>0.01%</i></span></div>
</div>
<div class="gauge-block">
<div class="gauge-label">
<span>I<sub>ctx</sub> <span class="dim">how much the document moved the model, in nats</span></span>
<strong id="r-ictx"></strong>
</div>
<div class="gauge slim">
<div class="zone z-inert" style="width:10%"></div><div class="zone z-partial" style="width:40%"></div><div class="zone z-strong" style="width:50%"></div>
<div id="g-ictx" class="marker"></div>
</div>
<div class="gauge-ticks"><span style="left:10%">0.05<br><i>inert</i></span><span style="left:50%">0.5<br><i>reshaping</i></span></div>
</div>
<div class="temp-block">
<div class="gauge-label">
<span>Sampling temperature <span class="dim">(how randomly the model picks its answer)</span></span>
<strong id="t-value">1.0</strong>
</div>
<input id="temp" type="range" min="0.2" max="2.0" step="0.1" value="1.0">
<p id="temp-readout" class="dim small"></p>
</div>
<details class="fine">
<summary>Measurement details</summary>
<table class="kv">
<tr><td>P(answer | query alone)</td><td id="r-p0"></td></tr>
<tr><td>P(answer | context + query)</td><td id="r-p1"></td></tr>
<tr><td>Answer tokenization</td><td id="r-tok"></td></tr>
<tr><td>D<sub>sync</sub> definition</td><td>−ln P(answer given context and query): the negative natural logarithm of the correct answer's probability</td></tr>
<tr><td>I<sub>ctx</sub> definition</td><td>Kullback-Leibler divergence between the with-document and without-document distributions, over all 50,257 tokens</td></tr>
</table>
</details>
</div>
</section>
<!-- ================= GUIDE ================= -->
<section id="guide" class="panel">
<h2>The five stages</h2>
<table class="stages">
<tr><td><i class="dot d1"></i><strong>Acquire</strong></td><td>Pre-training compresses a corpus into weights.</td><td class="fail">Facts stored with no record of when or where they were learned.</td></tr>
<tr><td><i class="dot d2"></i><strong>Store</strong></td><td>Facts live in weights, external indices, or both.</td><td class="fail">The two copies of a fact age independently.</td></tr>
<tr><td><i class="dot d3"></i><strong>Retrieve</strong></td><td>Attention recalls; RAG fetches documents into context.</td><td class="fail">What was retrieved may not control the output.</td></tr>
<tr><td><i class="dot d4"></i><strong>Update</strong></td><td>Weights edited, indices refreshed, conflicts resolved.</td><td class="fail">Edits damage neighbors; index updates leave weights stale.</td></tr>
<tr><td><i class="dot d5"></i><strong>Forget</strong></td><td>Deliberate unlearning; accidental forgetting and eviction.</td><td class="fail">Removed facts recoverable; retained facts damaged.</td></tr>
</table>
<div class="alert a-note">
<span class="alert-title">Note</span>
The probe measures the boundary between <i class="dot d3"></i>Retrieve and
<i class="dot d4"></i>Update: the document arrives, and the model must decide
which copy of the fact to trust.
</div>
<h2>The diagnosis grid</h2>
<table class="grid2">
<tr><th></th><th>I<sub>ctx</sub> low</th><th>I<sub>ctx</sub> high</th></tr>
<tr><th>D<sub>sync</sub> low</th><td class="c-ok">fact never conflicted</td><td class="c-ok">context resolved the conflict</td></tr>
<tr><th>D<sub>sync</sub> high</th><td class="c-fail">resolution failure: context ignored</td><td class="c-drift">drift: context influential but losing</td></tr>
</table>
<h2>The three presets</h2>
<table>
<tr><td><span class="chip chip-fail">failure</span></td><td><strong>Vioxx withdrawal.</strong> The notice is in the prompt; the model still says "safe". Paper's fp32 run: D<sub>sync</sub> 12.05, I<sub>ctx</sub> 0.033.</td></tr>
<tr><td><span class="chip chip-drift">trap</span></td><td><strong>British monarch.</strong> The death announcement's strongest effect is boosting " Queen". Correct context reinforcing the wrong answer.</td></tr>
<tr><td><span class="chip chip-influence">drift</span></td><td><strong>Twitter rename.</strong> Context lifts the right answer by orders of magnitude and still loses to " Twitter".</td></tr>
</table>
<h2>Terms used on this page</h2>
<table class="glossary">
<tr><td><strong>Token</strong></td><td>The unit a model reads and writes. Roughly a word or word fragment; " withdrawn" is one token, and the leading space is part of it.</td></tr>
<tr><td><strong>Parametric memory</strong></td><td>What the model absorbed into its weights during training. Fixed after training, and the model cannot tell you when it learned any of it.</td></tr>
<tr><td><strong>Context</strong></td><td>What you put in the prompt right now, including any document retrieved for the model to read.</td></tr>
<tr><td><strong>Nat</strong></td><td>A unit of information, measured with natural logarithms. Here it converts directly to probability: a value of <em>n</em> nats means the correct answer holds probability e<sup>−n</sup>. One nat is roughly 37%, nine nats is roughly one in ten thousand.</td></tr>
<tr><td><strong>Surprisal</strong></td><td>How surprised the model is by an answer, written as −ln P. Low when the model expected it, high when it did not. D<sub>sync</sub> is the surprisal of the correct answer.</td></tr>
<tr><td><strong>KL divergence</strong></td><td>A measure of how far one probability distribution sits from another. I<sub>ctx</sub> uses it to ask whether the document changed the model's mind about anything at all.</td></tr>
<tr><td><strong>Temperature</strong></td><td>How randomly a model picks among candidate answers. Low values make it repeat its favorite; high values spread the choice out.</td></tr>
<tr><td><strong>Quantized</strong></td><td>Weights stored at reduced precision so the model downloads and runs faster. This page uses 8-bit weights, which shift individual probabilities slightly without changing any conclusion.</td></tr>
</table>
<h2>Build your own probe</h2>
<p>
Any fact change with a single-word answer works. Three rules make a clean probe: the query
ends mid-sentence so the next token is the answer; the document states the new fact
plainly; the answer is one word, because the estimator measures the first token. Leadership
changes, product renames, and policy reversals all fit.
</p>
<div class="alert a-warning">
<span class="alert-title">Scope</span>
GPT-2 base is small, open, and confound-free, which makes every number reproducible,
including in this browser. The browser build is 8-bit quantized: probabilities shift
relative to full precision, the regimes and conclusions do not. Exact fp32 values come
from the script in the repository. The estimator measures the first token of a
single-word answer under deterministic evaluation.
</div>
<h2>The poster</h2>
<p>
The whole argument on one page: the problem, the five-stage framework, the Vioxx
finding, the metric, and the proposed architecture.
<a href="poster.pdf">Open the full-resolution A0 PDF</a>.
</p>
<p style="text-align:center">
<a href="poster.pdf"><img src="poster-preview.png"
alt="A0 conference poster for The Knowledge Lifecycle of Large Language Models"
style="max-width:100%;width:660px;border:1px solid rgba(128,128,128,.35);border-radius:6px"></a>
</p>
</section>
</main>
<footer>
<div class="authors">
<div class="author">
<a href="https://github.com/Amey-Thakur"><img class="avatar" src="amey-thakur.jpg" alt="Amey Thakur"></a>
<p><a href="https://github.com/Amey-Thakur"><strong>Amey Thakur</strong></a></p>
<p><a href="https://orcid.org/0000-0001-5644-1575"><img src="https://img.shields.io/badge/ORCID-0000--0001--5644--1575-A6CE39" alt="ORCID"></a></p>
</div>
<div class="author">
<a href="https://github.com/sarveshtalele"><img class="avatar" src="https://github.com/sarveshtalele.png" alt="Sarvesh Talele"></a>
<p><a href="https://github.com/sarveshtalele"><strong>Sarvesh Talele</strong></a></p>
<p><a href="https://orcid.org/0009-0002-0818-461X"><img src="https://img.shields.io/badge/ORCID-0009--0002--0818--461X-A6CE39" alt="ORCID"></a></p>
</div>
</div>
<details>
<summary>Cite this work</summary>
<pre>@article{thakur2026lifecycle,
author = {Thakur, Amey and Talele, Sarvesh},
title = {The Knowledge Lifecycle of Large Language Models},
journal = {arXiv preprint},
year = {2026}
}</pre>
</details>
<p class="foot-links">
<a href="https://github.com/Amey-Thakur/LLM-KNOWLEDGE-LIFECYCLE">Paper repository</a>
· <a href="https://amey-thakur.github.io">Amey's Arc</a>
</p>
</footer>
<script type="module" src="main.js"></script>
</body>
</html>
|