Hugging Face's logo

Spaces:
luuuulinnnn
/
world-memory-deck
Running

App Files Community
world-memory-deck
File size: 29,000 Bytes
282daaf
 
 
 
 
d7df417
282daaf
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
d7df417
 
282daaf
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91bd53b
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>World Memory — Knownness-Conditioned Video Generation</title>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/reveal.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/theme/black.css" id="theme">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/plugin/highlight/monokai.css">
<style>
  :root{
    --bg0:#0a0e1a; --bg1:#0f1626; --ink:#e8ecf6; --muted:#9aa6c4;
    --teal:#35d0ba; --purple:#8b7cff; --amber:#ffb454; --coral:#ff6b81;
    --known:#35d0ba; --unknown:#ff6b9d; --line:#2a3550;
  }
  .reveal { font-family: "Inter","Helvetica Neue",Arial,sans-serif; color:var(--ink); font-weight:400; }
  .reveal .slides { text-align:left; }
  .reveal section { padding:0 12px; }
  .reveal h1,.reveal h2,.reveal h3,.reveal h4{ color:var(--ink); font-weight:700; letter-spacing:-.01em; text-transform:none; text-shadow:none; }
  .reveal h1{ font-size:2.35em; line-height:1.05; }
  .reveal h2{ font-size:1.65em; }
  .reveal h3{ font-size:1.15em; }
  .reveal p,.reveal li{ color:var(--ink); font-size:.72em; line-height:1.5; }
  .reveal a{ color:var(--teal); }
  .reveal em{ color:var(--amber); font-style:normal; }
  .reveal strong{ color:var(--purple); font-weight:700; }
  .reveal .slide-background{ background:radial-gradient(1200px 800px at 20% -10%, #16203a 0%, var(--bg1) 40%, var(--bg0) 100%); }
  .kicker{ color:var(--teal); font-size:.6em; font-weight:700; letter-spacing:.22em; text-transform:uppercase; margin:0 0 .5em; }
  .phase-tag{ display:inline-block; font-size:.5em; font-weight:700; letter-spacing:.15em; text-transform:uppercase;
    padding:.35em .8em; border-radius:999px; margin-bottom:.7em; }
  .p1{ background:rgba(53,208,186,.14); color:var(--teal); border:1px solid rgba(53,208,186,.4); }
  .p2{ background:rgba(139,124,255,.14); color:var(--purple); border:1px solid rgba(139,124,255,.4); }
  .p3{ background:rgba(255,180,84,.14); color:var(--amber); border:1px solid rgba(255,180,84,.4); }
  .p4{ background:rgba(255,107,129,.14); color:var(--coral); border:1px solid rgba(255,107,129,.4); }
  .cols{ display:flex; gap:26px; align-items:flex-start; }
  .col{ flex:1; }
  .card{ background:rgba(255,255,255,.035); border:1px solid var(--line); border-radius:14px; padding:16px 18px; }
  .card h3{ margin:.1em 0 .35em; font-size:1.05em; }
  .card p, .card li{ font-size:.66em; color:var(--muted); }
  .card .num{ font-size:.5em; font-weight:700; letter-spacing:.1em; color:var(--teal); }
  .reveal ul{ margin-left:1.05em; }
  .reveal li{ margin:.28em 0; }
  .muted{ color:var(--muted)!important; }
  .eq{ background:rgba(255,255,255,.04); border:1px solid var(--line); border-left:3px solid var(--teal);
    border-radius:10px; padding:12px 18px; font-family:"SFMono-Regular",Consolas,monospace; font-size:.8em; color:var(--ink); }
  .footer{ position:absolute; bottom:14px; left:24px; font-size:12px; color:var(--muted); letter-spacing:.05em; }
  .pageno{ position:absolute; bottom:14px; right:24px; font-size:12px; color:var(--muted); }
  .chip{ display:inline-block; font-size:.62em; padding:.25em .7em; margin:.15em .2em 0 0; border-radius:8px;
    background:rgba(255,255,255,.05); border:1px solid var(--line); color:var(--ink); }
  .big{ font-size:1.15em; }
  .lead{ font-size:.9em; color:var(--muted); max-width:22em; }
  .reveal table{ font-size:.62em; }
  .reveal td,.reveal th{ border-color:var(--line); padding:.4em .7em; }
  .reveal th{ color:var(--teal); }
  .gridnums{ display:flex; gap:18px; }
  .gridnums .stat{ text-align:center; background:rgba(255,255,255,.03); border:1px solid var(--line); border-radius:14px; padding:16px 10px; flex:1; }
  .gridnums .stat .v{ font-size:1.9em; font-weight:800; color:var(--teal); line-height:1; }
  .gridnums .stat .l{ font-size:.5em; color:var(--muted); letter-spacing:.05em; margin-top:.5em; }
  .figwrap{ width:100%; }
  .figwrap svg{ width:100%; height:auto; display:block; }
  .center{ text-align:center; }
  .titlewrap{ text-align:center; }
  .titlewrap .sub{ font-size:.85em; color:var(--muted); margin-top:.2em; }
  .authors{ margin-top:1.4em; color:var(--muted); font-size:.62em; letter-spacing:.04em; }
  .legendrow{ display:flex; gap:8px; align-items:center; font-size:.6em; color:var(--muted); }
  .swatch{ width:14px; height:14px; border-radius:4px; display:inline-block; }
</style>
</head>
<body>
<div class="reveal"><div class="slides">

<!-- 1. TITLE -->
<section data-auto-animate>
  <div class="titlewrap" style="margin-top:6%;">
    <p class="kicker">CoRL 2026 · Long-Video World Models</p>
    <h1>World&nbsp;Memory</h1>
    <p class="sub big"><strong style="color:var(--amber)">Knownness-Conditioned</strong> Long-Video Generation</p>
    <p class="lead" style="margin:1.1em auto 0;">A frozen video diffusion model that <em>reuses what it has seen</em> — and generates only what it hasn't.</p>
    <p class="authors">A 3D-grounded memory bank &nbsp;·&nbsp; per-token noise scheduling &nbsp;·&nbsp; write-back rollout</p>
  </div>
  <div class="footer">World Memory · working title</div>
</section>

<!-- 2. PROBLEM -->
<section>
  <p class="kicker">The problem</p>
  <h2>Video models <em>re-hallucinate</em> on revisit</h2>
  <div class="cols" style="margin-top:22px;">
    <div class="col">
      <p>Generate a long video where the camera moves away and comes <strong>back</strong> to a place it already saw.</p>
      <ul>
        <li>Frame-by-frame diffusion has <em>no memory of 3D</em>.</li>
        <li>On return, the same wall / table / sky is <em>re-invented differently</em> — flicker, drift, ghosting.</li>
        <li>Naïvely copying old pixels (splatting) leaves holes, seams, and stale geometry.</li>
      </ul>
    </div>
    <div class="col">
      <div class="card">
        <p class="num">GOAL</p>
        <h3>Revisit ⇒ recall, not re-generate</h3>
        <p>When the camera returns to a known place, the model should <strong>restore the same scene</strong> from memory — and still <strong>freely generate</strong> genuinely new regions.</p>
      </div>
      <div class="card" style="margin-top:14px;">
        <p class="num">KEY IDEA</p>
        <h3>One scalar per token: <span style="color:var(--teal)">knownness&nbsp;k</span></h3>
        <p>Route each space-time token between <em>reuse-and-repair</em> and <em>generate-from-noise</em> — inside a single frozen diffusion model.</p>
      </div>
    </div>
  </div>
  <div class="pageno">2</div>
</section>

<!-- 3. ONE-LINE IDEA -->
<section>
  <p class="kicker">The recipe in one line</p>
  <h2 style="margin-bottom:24px;">Memory sets the <span style="color:var(--teal)">noise level</span>; diffusion does the rest</h2>
  <div class="cols">
    <div class="col card" style="border-left:3px solid var(--teal);">
      <p class="num" style="color:var(--teal)">OFFLINE</p>
      <h3>Build a 3D world memory</h3>
      <p>Depth + poses → a persistent voxel point cloud. Reproject it into every target camera to get <strong>z_mem</strong> (what should be there), <strong>k</strong> (how sure), and <strong>Xtok</strong> (where in 3D).</p>
    </div>
    <div class="col card" style="border-left:3px solid var(--coral);">
      <p class="num" style="color:var(--coral)">ONLINE</p>
      <h3>Per-token noise from k</h3>
      <p>Wan DiT natively supports a <em>different diffusion timestep per token</em>. Known → low noise (repair). Unknown → high noise (generate). New output is <strong>written back</strong> into memory.</p>
    </div>
  </div>
  <div class="eq center" style="margin-top:26px;">high&nbsp;k &nbsp;→&nbsp; start near <span style="color:var(--teal)">z_mem</span> &nbsp;·&nbsp; short denoise &nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp; low&nbsp;k &nbsp;→&nbsp; start from <span style="color:var(--coral)">noise</span> &nbsp;·&nbsp; full generation</div>
  <div class="pageno">3</div>
</section>

<!-- 4. PIPELINE FIGURE -->
<section data-background-color="#080b14">
  <p class="kicker" style="margin-bottom:6px;">System overview</p>
  <div class="figwrap">
    <!-- ===================== PIPELINE SVG ===================== -->
    <svg viewBox="0 0 1720 760" xmlns="http://www.w3.org/2000/svg" font-family="Inter, Arial, sans-serif">
      <defs>
        <marker id="arrow" markerWidth="10" markerHeight="10" refX="7" refY="3" orient="auto" markerUnits="strokeWidth">
          <path d="M0,0 L7,3 L0,6 Z" fill="#9aa6c4"/>
        </marker>
        <marker id="arrowT" markerWidth="10" markerHeight="10" refX="7" refY="3" orient="auto" markerUnits="strokeWidth">
          <path d="M0,0 L7,3 L0,6 Z" fill="#35d0ba"/>
        </marker>
        <marker id="arrowC" markerWidth="10" markerHeight="10" refX="7" refY="3" orient="auto" markerUnits="strokeWidth">
          <path d="M0,0 L7,3 L0,6 Z" fill="#ff6b81"/>
        </marker>
        <linearGradient id="kbar" x1="0" y1="0" x2="1" y2="0">
          <stop offset="0" stop-color="#ff6b9d"/><stop offset="0.5" stop-color="#8b7cff"/><stop offset="1" stop-color="#35d0ba"/>
        </linearGradient>
        <linearGradient id="lanep1" x1="0" y1="0" x2="0" y2="1">
          <stop offset="0" stop-color="#35d0ba" stop-opacity="0.10"/><stop offset="1" stop-color="#35d0ba" stop-opacity="0.02"/>
        </linearGradient>
        <linearGradient id="lanep2" x1="0" y1="0" x2="0" y2="1">
          <stop offset="0" stop-color="#ff6b81" stop-opacity="0.10"/><stop offset="1" stop-color="#ff6b81" stop-opacity="0.02"/>
        </linearGradient>
        <style>
          .lbl{ fill:#e8ecf6; font-size:15px; font-weight:600; }
          .sub{ fill:#9aa6c4; font-size:12px; }
          .hdr{ font-size:13px; font-weight:800; letter-spacing:2px; }
          .box{ fill:#111a2e; stroke:#2a3550; stroke-width:1.5; }
          .flow{ stroke:#9aa6c4; stroke-width:2; fill:none; }
          .flowT{ stroke:#35d0ba; stroke-width:2; fill:none; }
          .tiny{ fill:#9aa6c4; font-size:11px; }
        </style>
      </defs>

      <!-- lane backgrounds -->
      <rect x="278" y="70" width="500" height="600" rx="18" fill="url(#lanep1)" stroke="#35d0ba" stroke-opacity="0.25"/>
      <rect x="1052" y="70" width="470" height="600" rx="18" fill="url(#lanep2)" stroke="#ff6b81" stroke-opacity="0.25"/>

      <!-- lane headers -->
      <text x="150" y="52" class="hdr" fill="#9aa6c4" text-anchor="middle">INPUT</text>
      <text x="528" y="52" class="hdr" fill="#35d0ba" text-anchor="middle">PHASE 1 · OFFLINE WORLD MEMORY</text>
      <text x="915" y="52" class="hdr" fill="#8b7cff" text-anchor="middle">MEMORY BUNDLE</text>
      <text x="1287" y="52" class="hdr" fill="#ff6b81" text-anchor="middle">PHASE 2 / 4 · ONLINE DiT</text>
      <text x="1632" y="52" class="hdr" fill="#9aa6c4" text-anchor="middle">OUTPUT</text>

      <!-- INPUT -->
      <rect class="box" x="40" y="300" width="220" height="150" rx="12"/>
      <text x="150" y="336" class="lbl" text-anchor="middle">Posed Video</text>
      <text x="150" y="358" class="sub" text-anchor="middle">832 × 480 · 81 frames</text>
      <!-- out-and-back glyph -->
      <path d="M70,410 C110,384 150,384 190,410" stroke="#35d0ba" stroke-width="2.5" fill="none"/>
      <path d="M190,410 C150,436 110,436 72,412" stroke="#ff6b81" stroke-width="2.5" fill="none" marker-end="url(#arrowC)"/>
      <circle cx="70" cy="410" r="4" fill="#e8ecf6"/>
      <text x="150" y="440" class="tiny" text-anchor="middle">out-and-back trajectory → forced revisit</text>

      <!-- PHASE 1 sub-boxes -->
      <rect class="box" x="308" y="92" width="440" height="70" rx="10"/>
      <text x="528" y="122" class="lbl" text-anchor="middle">Metric Depth  +  Global Scale Calibration</text>
      <text x="528" y="143" class="sub" text-anchor="middle">depth matched to camera baseline → 3D addresses only</text>

      <rect class="box" x="308" y="192" width="440" height="196" rx="10" stroke="#35d0ba" stroke-opacity="0.5"/>
      <text x="528" y="220" class="lbl" text-anchor="middle" fill="#35d0ba">PointMemory — world voxel point cloud</text>
      <text x="335" y="250" class="sub">• 3D position &amp; average RGB</text>
      <text x="335" y="272" class="sub">• observation count</text>
      <text x="335" y="294" class="sub">• per-view directions</text>
      <text x="335" y="316" class="sub">• color variance → ghosting signal</text>
      <text x="528" y="352" class="tiny" text-anchor="middle">fuse ~11 observed frames · never forgets</text>
      <rect x="335" y="360" width="386" height="16" rx="4" fill="#0a0e1a" stroke="#2a3550"/>
      <text x="528" y="372" class="tiny" text-anchor="middle">persistent across the whole rollout</text>

      <rect class="box" x="308" y="418" width="440" height="76" rx="10"/>
      <text x="528" y="448" class="lbl" text-anchor="middle">Reproject → target cameras</text>
      <text x="528" y="470" class="sub" text-anchor="middle">z-buffer · adaptive splat radius · depth test</text>

      <!-- internal phase-1 arrows -->
      <line class="flowT" x1="528" y1="162" x2="528" y2="192" marker-end="url(#arrowT)"/>
      <line class="flowT" x1="528" y1="388" x2="528" y2="418" marker-end="url(#arrowT)"/>

      <!-- BUNDLE boxes -->
      <rect class="box" x="800" y="150" width="230" height="72" rx="10" stroke="#8b7cff" stroke-opacity="0.5"/>
      <text x="915" y="180" class="lbl" text-anchor="middle" fill="#8b7cff">z_mem</text>
      <text x="915" y="202" class="sub" text-anchor="middle">VAE-encoded memory reprojection</text>

      <rect class="box" x="800" y="242" width="230" height="80" rx="10" stroke="#8b7cff" stroke-opacity="0.5"/>
      <text x="915" y="268" class="lbl" text-anchor="middle" fill="#8b7cff">knownness k</text>
      <rect x="828" y="282" width="174" height="12" rx="6" fill="url(#kbar)"/>
      <text x="828" y="312" class="tiny">unknown</text><text x="1002" y="312" class="tiny" text-anchor="end">known</text>

      <rect class="box" x="800" y="342" width="230" height="72" rx="10" stroke="#8b7cff" stroke-opacity="0.5"/>
      <text x="915" y="372" class="lbl" text-anchor="middle" fill="#8b7cff">Xtok</text>
      <text x="915" y="394" class="sub" text-anchor="middle">per-token 3D world position</text>

      <rect class="box" x="800" y="434" width="230" height="72" rx="10" stroke="#8b7cff" stroke-opacity="0.5"/>
      <text x="915" y="464" class="lbl" text-anchor="middle" fill="#8b7cff">memory slot bank</text>
      <text x="915" y="486" class="sub" text-anchor="middle">compact 3D key/value memory</text>

      <!-- DiT -->
      <rect class="box" x="1080" y="92" width="414" height="470" rx="12" stroke="#ff6b81" stroke-opacity="0.5"/>
      <text x="1287" y="122" class="lbl" text-anchor="middle" fill="#ff6b81">Frozen Wan Video DiT</text>
      <text x="1287" y="142" class="sub" text-anchor="middle">+ small zero-init trainable modules</text>

      <rect x="1104" y="160" width="366" height="66" rx="8" fill="#0d1526" stroke="#2a3550"/>
      <text x="1287" y="186" class="lbl" text-anchor="middle">① Per-token noise level from k</text>
      <text x="1287" y="207" class="tiny" text-anchor="middle">low k → high noise · high k → low noise</text>

      <rect x="1104" y="238" width="366" height="66" rx="8" fill="#0d1526" stroke="#2a3550"/>
      <text x="1287" y="264" class="lbl" text-anchor="middle">② 3D positional encoding (Xtok)</text>
      <text x="1287" y="285" class="tiny" text-anchor="middle">same world point = same identity under camera motion</text>

      <rect x="1104" y="316" width="366" height="70" rx="8" fill="#0d1526" stroke="#2a3550"/>
      <text x="1287" y="342" class="lbl" text-anchor="middle">③ Geometry cross-attn · every 5 blocks</text>
      <text x="1287" y="363" class="tiny" text-anchor="middle">Xtok queries slot bank · Rope3D relative embedding</text>

      <rect x="1104" y="398" width="366" height="60" rx="8" fill="#0d1526" stroke="#2a3550"/>
      <text x="1287" y="424" class="lbl" text-anchor="middle">④ Confidence-gated shortcut</text>
      <text x="1287" y="444" class="tiny" text-anchor="middle">pass memory where k high · suppress where k low</text>

      <rect x="1104" y="470" width="366" height="66" rx="8" fill="#0a0e1a" stroke="#35d0ba" stroke-opacity="0.4"/>
      <text x="1287" y="496" class="lbl" text-anchor="middle" fill="#35d0ba">Zero-init</text>
      <text x="1287" y="516" class="tiny" text-anchor="middle">at start, modules output 0 → behaves like base Wan</text>

      <!-- OUTPUT -->
      <rect class="box" x="1548" y="300" width="150" height="150" rx="12"/>
      <text x="1623" y="368" class="lbl" text-anchor="middle">Generated</text>
      <text x="1623" y="390" class="lbl" text-anchor="middle">Video</text>

      <!-- MAIN FLOW ARROWS -->
      <line class="flow" x1="260" y1="375" x2="306" y2="290" marker-end="url(#arrow)"/>
      <path class="flow" d="M748,300 C775,300 775,186 798,186" marker-end="url(#arrow)"/>
      <path class="flow" d="M748,340 C775,340 775,282 798,282" marker-end="url(#arrow)"/>
      <path class="flow" d="M748,400 C775,400 775,378 798,378" marker-end="url(#arrow)"/>
      <path class="flow" d="M748,455 C775,455 775,470 798,470" marker-end="url(#arrow)"/>

      <!-- bundle -> DiT -->
      <path class="flow" d="M1030,186 C1055,186 1058,193 1102,193" marker-end="url(#arrow)"/>
      <path class="flow" d="M1030,282 C1055,282 1058,271 1102,271" marker-end="url(#arrow)"/>
      <path class="flow" d="M1030,378 C1055,300 1060,300 1102,290" marker-end="url(#arrow)"/>
      <path class="flow" d="M1030,470 C1055,360 1060,352 1102,348" marker-end="url(#arrow)"/>

      <!-- DiT -> output -->
      <line class="flow" x1="1494" y1="360" x2="1546" y2="368" marker-end="url(#arrow)"/>

      <!-- WRITE-BACK LOOP -->
      <path d="M1623,450 C1623,700 528,720 528,510" stroke="#ff6b81" stroke-width="2.5" stroke-dasharray="7 6" fill="none" marker-end="url(#arrowC)"/>
      <rect x="880" y="690" width="300" height="30" rx="8" fill="#0a0e1a" stroke="#ff6b81" stroke-opacity="0.5"/>
      <text x="1030" y="710" class="lbl" text-anchor="middle" fill="#ff6b81" font-size="13">write-back: new frames → hypothesis points</text>
      <text x="440" y="540" class="tiny" fill="#ff6b81">unknown → known</text>
    </svg>
    <!-- =================== /PIPELINE SVG =================== -->
  </div>
  <div class="pageno">4</div>
</section>

<!-- 5. PHASE 1 -->
<section>
  <span class="phase-tag p1">Phase 1 · offline caching</span>
  <h2>Video → persistent 3D world memory</h2>
  <div class="cols" style="margin-top:18px;">
    <div class="col">
      <ul>
        <li><strong>Out-and-back</strong> trajectory (~41 out / ~40 back) <em>forces a revisit</em> — the return path is real GT to score consistency.</li>
        <li><strong>Metric depth + one global scale</strong> matched to the camera baseline. Depth is only a <em>3D address</em>, never copied as pixels.</li>
        <li>Back-project ~11 frames into <strong>PointMemory</strong>: RGB, count, view directions, <em>color variance</em> (ghosting detector).</li>
        <li>Reproject to each target cam (z-buffer, adaptive splat) → coverage / count / diversity / disagreement maps.</li>
      </ul>
    </div>
    <div class="col">
      <div class="card">
        <p class="num">SAVED PER CLIP → one .pt file</p>
        <p><span class="chip">x0 (GT video, VAE)</span><span class="chip">z_mem</span><span class="chip">knownness k</span><span class="chip">Xtok</span><span class="chip">3D slot bank</span><span class="chip">camera poses</span></p>
      </div>
      <div class="card" style="margin-top:14px;">
        <p class="num">WHY VARIANCE MATTERS</p>
        <p>Consistent color across views ⇒ reliable memory. High disagreement ⇒ occlusion / dynamics / geometry error / <em>ghosting</em> — and lowers k.</p>
      </div>
    </div>
  </div>
  <div class="pageno">5</div>
</section>

<!-- 6. KNOWNNESS FIELD -->
<section>
  <span class="phase-tag p1">Phase 1 · the knownness field</span>
  <h2>What makes a region <span style="color:var(--teal)">known</span>?</h2>
  <div class="eq center" style="margin:22px auto; font-size:.95em;">
    k &nbsp;=&nbsp; <span style="color:var(--teal)">cover</span> · exp(&minus;β·<span style="color:var(--coral)">disagree</span>) · clip( tanh(<span style="color:var(--purple)">count</span>/n₀) + 0.3·<span style="color:var(--amber)">diversity</span> )
  </div>
  <div class="cols">
    <div class="col card" style="border-top:3px solid var(--teal)"><p class="num" style="color:var(--teal)">COVER</p><p>Is the region reprojected from memory at all?</p></div>
    <div class="col card" style="border-top:3px solid var(--purple)"><p class="num" style="color:var(--purple)">COUNT</p><p>Seen many times?</p></div>
    <div class="col card" style="border-top:3px solid var(--amber)"><p class="num" style="color:var(--amber)">DIVERSITY</p><p>Seen from many viewpoints?</p></div>
    <div class="col card" style="border-top:3px solid var(--coral)"><p class="num" style="color:var(--coral)">DISAGREE</p><p>Do observations conflict? Penalized.</p></div>
  </div>
  <p class="muted center" style="margin-top:20px; font-size:.72em;">High k only when a region is <strong>covered</strong>, <strong>seen often</strong>, from <strong>many views</strong>, and observations <strong>agree</strong>.</p>
  <div class="pageno">6</div>
</section>

<!-- 7. PHASE 2 MODEL -->
<section>
  <span class="phase-tag p2">Phase 2 · model</span>
  <h2>Frozen Wan DiT + four small hooks</h2>
  <div class="cols" style="margin-top:14px;">
    <div class="col">
      <div class="card" style="border-left:3px solid var(--coral)"><p class="num" style="color:var(--coral)">① CORE</p><h3>Knownness → per-token timestep</h3><p>Wan natively supports a different diffusion timestep per token. Known = low noise (repair); unknown = high noise (generate). No surgery on the backbone.</p></div>
      <div class="card" style="margin-top:12px;border-left:3px solid var(--purple)"><p class="num" style="color:var(--purple)">② GEOMETRY</p><h3>3D positional encoding (Xtok)</h3><p>Each token knows its world position and viewing direction → the same world point stays the same identity as the camera moves.</p></div>
    </div>
    <div class="col">
      <div class="card" style="border-left:3px solid var(--teal)"><p class="num" style="color:var(--teal)">③ RETRIEVAL</p><h3>Geometry-keyed cross-attention</h3><p>Every 5 blocks, Xtok queries the slot bank with <strong>Rope3D</strong> relative embedding — retrieve the same content from any viewing angle.</p></div>
      <div class="card" style="margin-top:12px;border-left:3px solid var(--amber)"><p class="num" style="color:var(--amber)">④ GATING</p><h3>Confidence-gated shortcut</h3><p>Residual that passes more memory where k is high, damped where k is low — bad memory can't leak in.</p></div>
    </div>
  </div>
  <p class="center" style="margin-top:16px;"><span class="chip" style="border-color:var(--teal);color:var(--teal)">All new modules zero-initialized → day-0 behavior ≈ original Wan</span></p>
  <div class="pageno">7</div>
</section>

<!-- 8. PHASE 3 TRAINING -->
<section>
  <span class="phase-tag p3">Phase 3 · training</span>
  <h2>Anchor-mixture Diffusion Forcing</h2>
  <div class="cols" style="margin-top:16px;">
    <div class="col">
      <p>Per token: sample noise level <em>t</em>, pick a clean <strong>anchor</strong>, build the input on the model's familiar noise-to-image manifold:</p>
      <div class="eq" style="margin:14px 0;">x_t = (1 &minus; t)·anchor + t·ε</div>
      <ul>
        <li>Usually <strong>anchor = x₀</strong> (real video latent).</li>
        <li>For memory-covered tokens, ~<strong>40%</strong> use <strong>anchor = z_mem</strong> — but only when <em>t ≥ 0.3</em>.</li>
        <li>Model always receives the <em>true t</em>.</li>
      </ul>
    </div>
    <div class="col">
      <div class="card" style="border-left:3px solid var(--amber)">
        <p class="num" style="color:var(--amber)">THE TARGET IS ALWAYS x₀</p>
        <p>Even when the anchor is z_mem, the loss points at the <strong>real</strong> frame. So the model learns to <em>repair</em> blurry / holey / ghosted memory into truth — <strong>not to copy it</strong>.</p>
      </div>
      <div class="card" style="margin-top:14px;border-left:3px solid var(--coral)">
        <p class="num" style="color:var(--coral)">UNKNOWN REGIONS UP-WEIGHTED</p>
        <p>Loss weighting stops the model from becoming a mere <em>splat cleaner</em> — it must truly synthesize content absent from memory.</p>
      </div>
    </div>
  </div>
  <div class="pageno">8</div>
</section>

<!-- 9. PHASE 4 INFERENCE -->
<section>
  <span class="phase-tag p4">Phase 4 · inference &amp; rollout</span>
  <h2>Knownness → per-token start noise → write-back</h2>
  <div class="cols" style="margin-top:14px;">
    <div class="col">
      <div class="eq" style="margin-bottom:10px;">τ = clip( 1 &minus; λk, &nbsp;0.45, &nbsp;1 )</div>
      <ul>
        <li><strong>Known:</strong> x_τ = (1&minus;τ)·z_mem + τ·ε — start from memory + a little noise, <em>short</em> denoise path.</li>
        <li><strong>Unknown:</strong> x_τ ≈ ε — start from ~pure noise, <em>full</em> generation.</li>
        <li>Each token integrates from its own τ → 0 in a <strong>single diffusion pass</strong>.</li>
        <li>Decode latents → frames with the VAE decoder.</li>
      </ul>
    </div>
    <div class="col">
      <div class="card" style="border-left:3px solid var(--coral)">
        <p class="num" style="color:var(--coral)">WRITE-BACK · CHUNKED ROLLOUT</p>
        <p>Back-project new frames into PointMemory as <strong>hypothesis points</strong>. Long videos run chunk-by-chunk; each chunk grows the memory.</p>
        <p style="margin-top:8px;">When the camera <strong>returns</strong>, k is now high → it starts from the stored content and <em>recovers the same scene</em> instead of re-hallucinating.</p>
      </div>
    </div>
  </div>
  <div class="pageno">9</div>
</section>

<!-- 10. NUMBERS -->
<section>
  <p class="kicker">Concretely, on DL3DV</p>
  <h2 style="margin-bottom:22px;">Every space-time token gets memory, a 3D address, and a k</h2>
  <div class="gridnums">
    <div class="stat"><div class="v">832×480</div><div class="l">VIDEO RESOLUTION</div></div>
    <div class="stat"><div class="v">81</div><div class="l">FRAMES</div></div>
    <div class="stat"><div class="v">[16,21,60,104]</div><div class="l">VAE LATENT</div></div>
    <div class="stat"><div class="v">30×52</div><div class="l">TOKEN GRID</div></div>
    <div class="stat"><div class="v">32,760</div><div class="l">TOKENS = 21×30×52</div></div>
  </div>
  <p class="muted center" style="margin-top:24px; font-size:.75em;">Each of the <strong>32,760</strong> tokens carries its own memory content, 3D position, and knownness.</p>
  <div class="pageno">10</div>
</section>

<!-- 11. TAKEAWAY -->
<section data-background-color="#080b14">
  <p class="kicker">Takeaway</p>
  <h2 style="margin-bottom:20px;">Memory decides <span style="color:var(--teal)">how much to generate</span></h2>
  <div class="cols">
    <div class="col card" style="border-top:3px solid var(--teal)"><h3 style="color:var(--teal)">Phase 1</h3><p>Posed video → world memory content, per-region <strong>knownness</strong>, per-token <strong>3D address</strong>.</p></div>
    <div class="col card" style="border-top:3px solid var(--amber)"><h3 style="color:var(--amber)">Phase 3</h3><p>Teach the model to <strong>repair</strong> memory, never blindly copy it. Target is always the real frame.</p></div>
    <div class="col card" style="border-top:3px solid var(--coral)"><h3 style="color:var(--coral)">Phase 4</h3><p>Knownness → per-token noise. Reuse+repair known regions, generate unknown, <strong>write back</strong>.</p></div>
  </div>
  <div class="eq center" style="margin-top:26px; font-size:.9em;">Revisit the same place ⇒ recover it from the <span style="color:var(--teal)">same world memory</span> — not a fresh hallucination.</div>
  <p class="muted center" style="margin-top:22px; font-size:.7em;">World Memory · working title · CoRL 2026</p>
  <div class="pageno">11</div>
</section>

</div></div>
<script src="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/reveal.js"></script>
<script src="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/plugin/highlight/highlight.js"></script>
<script src="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/plugin/notes/notes.js"></script>
<script>
  Reveal.initialize({
    width: 1280, height: 720, margin: 0.045,
    minScale: 0.2, maxScale: 2.0,
    hash: true, slideNumber: false, transition: 'slide', transitionSpeed: 'default',
    controlsTutorial: true, center: false,
    plugins: [ RevealHighlight, RevealNotes ]
  });
</script>
</body>
</html>