Laborator commited on
Commit
bc3caba
·
verified ·
1 Parent(s): 22ba3a0

Rebuild as interactive static demo: live reveal-schedule lever drives the Brier curve; correct dataset label to ForecastBench (CC BY-SA 4.0)

Browse files
Files changed (6) hide show
  1. README.md +35 -15
  2. app.py +0 -331
  3. data.json +5 -1
  4. frequency_sweep.png +0 -3
  5. index.html +543 -0
  6. requirements.txt +0 -2
README.md CHANGED
@@ -3,30 +3,50 @@ title: Second Loop · 3 · Thin Channel
3
  emoji: 📡
4
  colorFrom: green
5
  colorTo: gray
6
- sdk: gradio
7
- sdk_version: 5.0.0
8
- python_version: "3.12"
9
- app_file: app.py
10
  pinned: true
11
  license: mit
12
  short_description: Rare external truth before calibration collapses
13
  ---
14
 
15
- # Thin Channel demo Space
16
 
17
- Static visualization of Experiment 6 (`exp6-forecast-calibration`) of the
18
  [Second Loop](https://github.com/SergheiBrinza/thin-channel) project.
19
 
20
- This Space loads no model. All numbers, the per-schedule table and the
21
- plot are exact outputs of the live experimental run, bundled into
22
- `data.json` and `frequency_sweep.png`.
23
 
24
- Headline numbers:
25
- - 11 reveal schedules tested (day → year + never)
26
- - Corridor of 0.205 0.220 Brier on hold-out (any finite schedule)
27
- - 0.298 Brier on `never` collapses bit-for-bit to the raw 3B baseline
28
- - The cliff is binary: channel thickness is total signal volume, not
29
- the frequency at which it arrives
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  Source code, raw per-schedule JSON results, and methodology document:
32
  <https://github.com/SergheiBrinza/thin-channel>
 
3
  emoji: 📡
4
  colorFrom: green
5
  colorTo: gray
6
+ sdk: static
7
+ app_file: index.html
 
 
8
  pinned: true
9
  license: mit
10
  short_description: Rare external truth before calibration collapses
11
  ---
12
 
13
+ # Thin Channel interactive demo
14
 
15
+ Interactive visualization of Experiment 6 (`exp6-forecast-calibration`) of the
16
  [Second Loop](https://github.com/SergheiBrinza/thin-channel) project.
17
 
18
+ This Space loads **no model**. Everything is a static page driven by `data.json`
19
+ the verbatim aggregate output of the live experimental run.
 
20
 
21
+ ## The exhibit
22
+
23
+ Move the lever *how often the system gets to check reality* — across its 11
24
+ settings, from **every day** to **never**. The Brier curve redraws live and the
25
+ highlighted point tracks your setting. Drive the lever all the way to **never**
26
+ and the point jumps off the corridor up to **0.298** — the system collapses
27
+ bit-for-bit back to the raw 3B baseline. That is the whole finding, in one
28
+ gesture.
29
+
30
+ ## What the numbers say
31
+
32
+ - 11 reveal schedules tested (day → year + never).
33
+ - Corridor of **0.205 – 0.220** Brier on hold-out for *any* finite schedule.
34
+ - **0.298** on `never` — collapses to the raw 3B baseline.
35
+ - The cliff is binary: channel thickness is total signal volume across the
36
+ system's lifetime, not the frequency at which it arrives.
37
+ - Honest caveat (shown in the page): the raw 3B is useless as a forecaster on
38
+ its own; the `(c) ≈ (b)` ablation shows the entire calibration gain comes from
39
+ per-topic base rates, not from the model learning to predict.
40
+
41
+ ## Data and attribution
42
+
43
+ The forecasting questions come from **ForecastBench** (Karger, Bastani,
44
+ Yueh-Han, Jacobs, Halawi, Zhang & Tetlock, *ForecastBench: A Dynamic Benchmark
45
+ of AI Forecasting Capabilities*, ICLR 2025, arXiv:2409.19839), licensed
46
+ **CC BY-SA 4.0** —
47
+ [forecastingresearch/forecastbench-datasets](https://huggingface.co/datasets/forecastingresearch/forecastbench-datasets).
48
+ No raw market questions or per-question outcomes are redistributed here — only
49
+ aggregate Brier results bundled in `data.json`. Demo code: MIT.
50
 
51
  Source code, raw per-schedule JSON results, and methodology document:
52
  <https://github.com/SergheiBrinza/thin-channel>
app.py DELETED
@@ -1,331 +0,0 @@
1
- """Thin Channel demo Space — static visualization of Experiment 6
2
- of the Second Loop project.
3
-
4
- No model is loaded. No re-evaluation is performed. All numbers come from
5
- the JSON outputs of the live experimental run, bundled into data.json
6
- (and one png plot from the run).
7
- """
8
-
9
- from __future__ import annotations
10
-
11
- import json
12
- from pathlib import Path
13
-
14
- import gradio as gr
15
-
16
- HERE = Path(__file__).resolve().parent
17
- DATA = json.loads((HERE / "data.json").read_text())
18
- META = DATA["meta"]
19
- SCHEDULES = DATA["schedules"]
20
- ABL = DATA["ablations"]
21
- PLOT_PATH = "frequency_sweep.png"
22
-
23
- REPO_URL = META.get("repo", "https://github.com/SergheiBrinza/thin-channel")
24
-
25
-
26
- # Compute the corridor and never gap for the headline numbers
27
- FINITE = [s for s in SCHEDULES if s["period_days"] is not None]
28
- NEVER = next(s for s in SCHEDULES if s["period_days"] is None)
29
- CORR_MIN = min(s["holdout_brier_final"] for s in FINITE)
30
- CORR_MAX = max(s["holdout_brier_final"] for s in FINITE)
31
-
32
-
33
- def render_schedule_table() -> str:
34
- rows = []
35
- for s in SCHEDULES:
36
- name = s["name"]
37
- period = "∞" if s["period_days"] is None else f"{s['period_days']}d"
38
- revealed = int(s["total_revealed"])
39
- b_c = s["holdout_brier_final"]
40
- b_b = s["holdout_brier_base_only"]
41
- b_a = s["holdout_brier_raw"]
42
- is_never = s["period_days"] is None
43
- cls = "schedule-row never-row" if is_never else "schedule-row"
44
- rows.append(f"""
45
- <tr class="{cls}">
46
- <td class="td-name">{name}</td>
47
- <td class="td-period">{period}</td>
48
- <td class="td-revealed">{revealed}</td>
49
- <td class="td-brier td-brier-c">{b_c:.4f}</td>
50
- <td class="td-brier td-brier-b">{b_b:.4f}</td>
51
- <td class="td-brier td-brier-a">{b_a:.4f}</td>
52
- </tr>""")
53
- return f"""
54
- <table class="schedule-table">
55
- <thead>
56
- <tr>
57
- <th>Frequency</th><th>Period</th><th>Revealed</th>
58
- <th>(c) mix</th><th>(b) base only</th><th>(a) raw 3B</th>
59
- </tr>
60
- </thead>
61
- <tbody>{''.join(rows)}</tbody>
62
- </table>
63
- """
64
-
65
-
66
- CSS = """
67
- @import url('https://fonts.googleapis.com/css2?family=Playfair+Display:ital,wght@1,700&family=Inter:wght@400;500;700;800&family=JetBrains+Mono:wght@500;700&display=swap');
68
-
69
- :root {
70
- --bg: #000000; --bg-card: #0A0A0A; --bg-elev: #141414;
71
- --border: #1F1F1F; --border-strong: #2A2A2A;
72
- --text: #FFFFFF; --text-mute: #A8A8A8; --text-dim: #6B6B6B;
73
- --gold: #D4AF37; --gold-hi: #E8C84A;
74
- --red: #FF2A2A; --green: #1FD160;
75
- --mono: 'JetBrains Mono', ui-monospace, monospace;
76
- --serif: 'Playfair Display', serif;
77
- --sans: 'Inter', system-ui, sans-serif;
78
- }
79
-
80
- .gradio-container, body {
81
- background: var(--bg) !important; color: var(--text) !important;
82
- font-family: var(--sans) !important;
83
- max-width: 1180px !important; margin: 0 auto !important;
84
- padding: 20px 24px 60px 24px !important;
85
- }
86
- footer, .built-with, .show-api { display: none !important; }
87
-
88
- .head {
89
- border: 1px solid var(--border-strong); border-radius: 14px;
90
- padding: 28px 32px 24px 32px;
91
- background: linear-gradient(180deg, #0A0A0A 0%, #050505 100%);
92
- }
93
- .head-top { display: flex; align-items: flex-start; justify-content: space-between;
94
- gap: 24px; padding-bottom: 18px; border-bottom: 1px solid var(--border);
95
- margin-bottom: 16px; }
96
- .head-brand { display: flex; align-items: center; gap: 16px; }
97
- .head-icon { width: 56px; height: 56px; border: 1px solid var(--border-strong);
98
- border-radius: 12px; display: flex; align-items: center; justify-content: center;
99
- font-size: 28px; background: radial-gradient(60% 60% at 50% 40%, #1A1A1A 0%, #050505 100%); }
100
- .head-title { font-family: var(--serif); font-style: italic; font-weight: 700;
101
- font-size: 36px; color: var(--text); line-height: 1; margin: 2px 0 6px 0; }
102
- .head-subtitle { font-family: var(--mono); font-size: 12px; letter-spacing: 0.16em;
103
- color: var(--text-mute); text-transform: uppercase; }
104
- .head-right { text-align: right; }
105
- .submitted-label { font-family: var(--mono); font-size: 10px; letter-spacing: 0.22em;
106
- color: var(--text-dim); text-transform: uppercase; display: block; margin-bottom: 4px; }
107
- .submitted-name { font-family: var(--serif); font-style: italic; font-weight: 700;
108
- font-size: 22px; color: var(--text); }
109
- .status-pill { display: inline-flex; align-items: center; gap: 6px; margin-top: 10px;
110
- padding: 5px 12px; border-radius: 999px;
111
- background: rgba(31, 209, 96, 0.08); border: 1px solid rgba(31, 209, 96, 0.5);
112
- font-family: var(--mono); font-size: 10px; letter-spacing: 0.18em;
113
- color: var(--green); text-transform: uppercase; }
114
- .status-dot { width: 7px; height: 7px; border-radius: 50%; background: var(--green); }
115
-
116
- .head-tag { text-align: center; margin: 4px 0 18px 0;
117
- font-family: var(--mono); font-size: 12px; letter-spacing: 0.22em;
118
- color: var(--gold); text-transform: uppercase; }
119
-
120
- .head-meta { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px 24px; }
121
- .head-meta .item { display: flex; align-items: baseline; gap: 8px; }
122
- .head-meta .k { font-family: var(--mono); font-size: 9.5px; letter-spacing: 0.2em;
123
- color: var(--text-dim); text-transform: uppercase; }
124
- .head-meta .v { font-family: var(--sans); font-size: 14px; font-weight: 700; color: var(--text); }
125
-
126
- .cta-wrap { text-align: center; margin: 24px 0 28px 0; }
127
- .cta { display: inline-flex; align-items: center; gap: 12px;
128
- padding: 16px 32px; background: var(--red); color: #fff;
129
- border-radius: 999px; border: 1px solid var(--red);
130
- font-family: var(--mono); font-size: 13px; letter-spacing: 0.18em;
131
- text-transform: uppercase; font-weight: 700; text-decoration: none;
132
- box-shadow: 0 0 30px rgba(255, 53, 53, 0.35);
133
- transition: transform 120ms ease, box-shadow 120ms ease; }
134
- .cta:hover { transform: translateY(-1px); box-shadow: 0 0 40px rgba(255, 53, 53, 0.55); }
135
- .cta-icon { width: 22px; height: 22px; border-radius: 999px; background: #fff;
136
- color: var(--red); display: flex; align-items: center; justify-content: center;
137
- font-size: 11px; }
138
-
139
- /* HEADLINE numbers */
140
- .summary { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px;
141
- margin-bottom: 22px; }
142
- .sum-card { background: var(--bg-card); border: 1px solid var(--border-strong);
143
- border-radius: 12px; padding: 18px 20px; }
144
- .sum-card .label { font-family: var(--mono); font-size: 9.5px; letter-spacing: 0.2em;
145
- color: var(--text-dim); text-transform: uppercase; margin-bottom: 8px; }
146
- .sum-card .value { font-family: var(--serif); font-style: italic; font-weight: 700;
147
- font-size: 28px; color: var(--text); }
148
- .sum-card .sub { font-family: var(--mono); font-size: 10px; letter-spacing: 0.12em;
149
- color: var(--text-mute); text-transform: uppercase; margin-top: 4px; }
150
- .sum-card.red-frame { border-color: rgba(255, 42, 42, 0.5); box-shadow: 0 0 20px rgba(255, 42, 42, 0.12); }
151
- .sum-card.green-frame { border-color: rgba(31, 209, 96, 0.5); box-shadow: 0 0 20px rgba(31, 209, 96, 0.12); }
152
-
153
- /* PLOT */
154
- .plot-wrap { background: var(--bg-card); border: 1px solid var(--border-strong);
155
- border-radius: 14px; padding: 22px; margin-bottom: 22px;
156
- text-align: center; }
157
- .plot-wrap img { max-width: 100%; height: auto; border-radius: 8px;
158
- background: #fff; }
159
- .plot-caption { font-family: var(--mono); font-size: 10px; letter-spacing: 0.16em;
160
- color: var(--text-dim); text-transform: uppercase; margin-top: 12px; }
161
-
162
- /* TABLE */
163
- .table-wrap { background: var(--bg-card); border: 1px solid var(--border-strong);
164
- border-radius: 14px; padding: 22px; }
165
- .schedule-table { width: 100%; border-collapse: collapse;
166
- font-family: var(--mono); font-size: 13px; }
167
- .schedule-table th { font-family: var(--mono); font-size: 9.5px; letter-spacing: 0.2em;
168
- color: var(--text-dim); text-transform: uppercase;
169
- text-align: right; padding: 8px 12px;
170
- border-bottom: 1px solid var(--border-strong); }
171
- .schedule-table th:first-child { text-align: left; }
172
- .schedule-table td { padding: 10px 12px; border-bottom: 1px solid var(--border);
173
- text-align: right; }
174
- .td-name { text-align: left !important; font-family: var(--serif); font-style: italic; font-weight: 700;
175
- font-size: 15px; color: var(--text); }
176
- .td-period { color: var(--text-mute); }
177
- .td-revealed { color: var(--text-mute); }
178
- .td-brier { color: var(--text); font-weight: 700; }
179
- .td-brier-c { color: var(--green); }
180
- .td-brier-b { color: var(--gold); }
181
- .td-brier-a { color: var(--red); opacity: 0.75; }
182
- .never-row { background: rgba(255, 42, 42, 0.06); }
183
- .never-row .td-brier-c { color: var(--red) !important; font-size: 14px; }
184
-
185
- /* ABLATION strip */
186
- .ablations { display: grid; grid-template-columns: 1fr 1fr 1fr; gap: 12px;
187
- margin: 22px 0; }
188
- .abl-card { background: var(--bg-card); border: 1px solid var(--border-strong);
189
- border-radius: 12px; padding: 18px 20px; text-align: center; }
190
- .abl-card .key { font-family: var(--mono); font-size: 10px; letter-spacing: 0.18em;
191
- color: var(--text-dim); text-transform: uppercase; margin-bottom: 8px; }
192
- .abl-card .val { font-family: var(--serif); font-style: italic; font-weight: 700;
193
- font-size: 32px; color: var(--text); }
194
- .abl-card .lbl { font-family: var(--mono); font-size: 10px; letter-spacing: 0.12em;
195
- color: var(--text-mute); text-transform: uppercase; margin-top: 4px; }
196
-
197
- .foot { margin-top: 32px; padding: 22px 24px;
198
- border: 1px solid var(--border); border-radius: 12px; background: var(--bg-card); }
199
- .foot .title { font-family: var(--mono); font-size: 10px; letter-spacing: 0.2em;
200
- color: var(--text-dim); text-transform: uppercase; margin-bottom: 10px; }
201
- .foot .body { font-family: var(--sans); font-size: 13px; color: var(--text-mute); line-height: 1.6; }
202
- .foot a { color: var(--gold); text-decoration: none; border-bottom: 1px dotted var(--gold); }
203
- .foot a:hover { color: var(--gold-hi); }
204
-
205
- .gradio-container .block { background: transparent !important; border: none !important; padding: 0 !important; }
206
- .gradio-container .form { background: transparent !important; border: none !important; }
207
- """
208
-
209
-
210
- def header_html() -> str:
211
- return f"""
212
- <div class="head">
213
- <div class="head-top">
214
- <div class="head-brand">
215
- <div class="head-icon">📡</div>
216
- <div>
217
- <div class="head-title">Thin Channel</div>
218
- <div class="head-subtitle">How rarely external truth can arrive before calibration collapses</div>
219
- </div>
220
- </div>
221
- <div class="head-right">
222
- <span class="submitted-label">Submitted by</span>
223
- <div class="submitted-name">{META['author']}</div>
224
- <div class="status-pill"><span class="status-dot"></span>Static demo</div>
225
- </div>
226
- </div>
227
- <div class="head-tag">★ &nbsp;Second Loop · Part 3 of 3&nbsp; ★</div>
228
- <div class="head-meta">
229
- <div class="item"><span class="k">Subject</span><span class="v">Qwen2.5-3B-Instruct</span></div>
230
- <div class="item"><span class="k">Dataset</span><span class="v">GSM-Plus / 1,646 Qs</span></div>
231
- <div class="item"><span class="k">Hold-out</span><span class="v">N = {META['n_holdout']}</span></div>
232
- <div class="item"><span class="k">License</span><span class="v">{META['license']}</span></div>
233
- </div>
234
- </div>
235
-
236
- <div class="cta-wrap">
237
- <a class="cta" href="{REPO_URL}" target="_blank" rel="noopener">
238
- <span class="cta-icon">▶</span> View the full repository on GitHub
239
- </a>
240
- </div>
241
-
242
- <div class="summary">
243
- <div class="sum-card">
244
- <div class="label">Schedules tested</div>
245
- <div class="value">11</div>
246
- <div class="sub">day → year + never</div>
247
- </div>
248
- <div class="sum-card green-frame">
249
- <div class="label">Corridor (any finite)</div>
250
- <div class="value">{CORR_MIN:.3f}–{CORR_MAX:.3f}</div>
251
- <div class="sub">Brier on hold-out</div>
252
- </div>
253
- <div class="sum-card red-frame">
254
- <div class="label">Never (no signal)</div>
255
- <div class="value">{NEVER['holdout_brier_final']:.3f}</div>
256
- <div class="sub">collapses to raw 3B</div>
257
- </div>
258
- <div class="sum-card">
259
- <div class="label">Cliff direction</div>
260
- <div class="value">Binary</div>
261
- <div class="sub">channel = volume, not freq</div>
262
- </div>
263
- </div>
264
-
265
- <div class="ablations">
266
- <div class="abl-card">
267
- <div class="key">(a) raw 3B only</div>
268
- <div class="val">{ABL['brier_a_raw']:.3f}</div>
269
- <div class="lbl">Brier on hold-out</div>
270
- </div>
271
- <div class="abl-card">
272
- <div class="key">(b) topic base rate only</div>
273
- <div class="val">{ABL['brier_b_base_only']:.3f}</div>
274
- <div class="lbl">no model needed</div>
275
- </div>
276
- <div class="abl-card">
277
- <div class="key">(c) mix (a + b)</div>
278
- <div class="val">{ABL['brier_c_mix']:.3f}</div>
279
- <div class="lbl">c − b = {ABL['delta_c_vs_b']:+.4f}</div>
280
- </div>
281
- </div>
282
-
283
- <div class="plot-wrap">
284
- <img src="/gradio_api/file=frequency_sweep.png" alt="Frequency sweep — Brier vs reveal period"/>
285
- <div class="plot-caption">Frequency sweep — committed plot from the experimental run</div>
286
- </div>
287
-
288
- <div class="table-wrap">
289
- {render_schedule_table()}
290
- </div>
291
- """
292
-
293
-
294
- def footer_html() -> str:
295
- return f"""
296
- <div class="foot">
297
- <div class="title">About this demo</div>
298
- <div class="body">
299
- Stage E of Experiment 7 in spirit, fully self-contained as Experiment 6:
300
- take 1,646 resolved binary forecasting questions, hide outcomes, reveal
301
- them on a schedule, fit a per-topic Beta calibration on revealed
302
- outcomes, measure Brier on hold-out. Eleven reveal schedules from
303
- "every day" to "never". The result is a binary cliff: any contact
304
- holds the corridor at <b>{CORR_MIN:.2f}–{CORR_MAX:.2f}</b>, zero contact
305
- collapses to <b>{NEVER['holdout_brier_final']:.2f}</b> (the raw 3B level).
306
- The c ≈ b ablation also shows that the entire calibration gain comes
307
- from per-topic base rates, not from any forecasting skill of the
308
- frozen 3B model — what is measured is the rate of signal accumulation,
309
- not learning to predict.
310
- <br/><br/>
311
- No live model is loaded in this Space: every number and the plot are
312
- verbatim outputs of the original run, bundled into <code>data.json</code>
313
- and <code>frequency_sweep.png</code>.
314
- <br/><br/>
315
- Full code, raw per-schedule JSON results, and methodology document:
316
- <a href="{REPO_URL}" target="_blank" rel="noopener">{REPO_URL}</a>.
317
- </div>
318
- </div>
319
- """
320
-
321
-
322
- with gr.Blocks(css=CSS, theme=gr.themes.Base(), title="Thin Channel · Second Loop") as demo:
323
- gr.HTML(header_html())
324
- gr.HTML(footer_html())
325
-
326
-
327
- if __name__ == "__main__":
328
- import os
329
- port = int(os.environ.get("GRADIO_SERVER_PORT", 7860))
330
- demo.launch(server_name="0.0.0.0", server_port=port,
331
- allowed_paths=[str(HERE / "frequency_sweep.png")])
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
data.json CHANGED
@@ -1,7 +1,11 @@
1
  {
2
  "meta": {
3
  "subject_model": "Qwen/Qwen2.5-3B-Instruct",
4
- "dataset": "qintongli/GSM-Plus (1,646 resolved binary forecasts)",
 
 
 
 
5
  "K": 10,
6
  "train_frac": 0.8,
7
  "n_total": 1646,
 
1
  {
2
  "meta": {
3
  "subject_model": "Qwen/Qwen2.5-3B-Instruct",
4
+ "dataset": "ForecastBench — forecastingresearch/forecastbench-datasets (1,646 resolved binary forecast questions, Jul 2024 – Dec 2025)",
5
+ "dataset_short": "ForecastBench",
6
+ "dataset_license": "CC BY-SA 4.0",
7
+ "dataset_url": "https://huggingface.co/datasets/forecastingresearch/forecastbench-datasets",
8
+ "dataset_paper": "Karger et al., ForecastBench, ICLR 2025 — arXiv:2409.19839",
9
  "K": 10,
10
  "train_frac": 0.8,
11
  "n_total": 1646,
frequency_sweep.png DELETED

Git LFS Details

  • SHA256: 61dac996668a2278bb07760a2e600d10e6ca0af0f92af602c2b2d5f92ae15150
  • Pointer size: 131 Bytes
  • Size of remote file: 217 kB
index.html ADDED
@@ -0,0 +1,543 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
6
+ <title>Thin Channel · Second Loop</title>
7
+ <link rel="preconnect" href="https://fonts.googleapis.com" />
8
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
9
+ <link href="https://fonts.googleapis.com/css2?family=Playfair+Display:ital,wght@1,700&family=Inter:wght@400;500;700;800&family=JetBrains+Mono:wght@500;700&display=swap" rel="stylesheet" />
10
+ <script src="https://cdn.plot.ly/plotly-2.35.2.min.js" charset="utf-8"></script>
11
+ <style>
12
+ :root{
13
+ --bg:#000000; --bg-card:#0A0A0A; --bg-elev:#141414;
14
+ --border:#1F1F1F; --border-strong:#2A2A2A;
15
+ --text:#FFFFFF; --text-mute:#A8A8A8; --text-dim:#6B6B6B;
16
+ --gold:#D4AF37; --gold-hi:#E8C84A;
17
+ --red:#FF2A2A; --green:#1FD160;
18
+ --mono:'JetBrains Mono',ui-monospace,monospace;
19
+ --serif:'Playfair Display',serif;
20
+ --sans:'Inter',system-ui,sans-serif;
21
+ }
22
+ *{box-sizing:border-box;}
23
+ html,body{margin:0;background:var(--bg);color:var(--text);font-family:var(--sans);}
24
+ .wrap{max-width:1180px;margin:0 auto;padding:22px 24px 64px 24px;}
25
+ a{color:var(--gold);text-decoration:none;border-bottom:1px dotted var(--gold);}
26
+ a:hover{color:var(--gold-hi);}
27
+
28
+ /* ---------- header ---------- */
29
+ .head{border:1px solid var(--border-strong);border-radius:14px;
30
+ padding:26px 30px 22px 30px;background:linear-gradient(180deg,#0A0A0A 0%,#050505 100%);}
31
+ .head-top{display:flex;align-items:flex-start;justify-content:space-between;gap:24px;
32
+ padding-bottom:16px;border-bottom:1px solid var(--border);margin-bottom:16px;}
33
+ .head-brand{display:flex;align-items:center;gap:16px;}
34
+ .head-icon{width:52px;height:52px;border:1px solid var(--border-strong);border-radius:12px;
35
+ display:flex;align-items:center;justify-content:center;
36
+ background:radial-gradient(60% 60% at 50% 40%,#1A1A1A 0%,#050505 100%);}
37
+ .head-title{font-family:var(--serif);font-style:italic;font-weight:700;font-size:34px;
38
+ color:var(--text);line-height:1;margin:2px 0 7px 0;}
39
+ .head-subtitle{font-family:var(--mono);font-size:11.5px;letter-spacing:.14em;
40
+ color:var(--text-mute);text-transform:uppercase;}
41
+ .head-right{text-align:right;white-space:nowrap;}
42
+ .submitted-label{font-family:var(--mono);font-size:10px;letter-spacing:.22em;
43
+ color:var(--text-dim);text-transform:uppercase;display:block;margin-bottom:4px;}
44
+ .submitted-name{font-family:var(--serif);font-style:italic;font-weight:700;font-size:20px;color:var(--text);}
45
+ .status-pill{display:inline-flex;align-items:center;gap:6px;margin-top:10px;padding:5px 12px;
46
+ border-radius:999px;background:rgba(31,209,96,.08);border:1px solid rgba(31,209,96,.5);
47
+ font-family:var(--mono);font-size:10px;letter-spacing:.18em;color:var(--green);text-transform:uppercase;}
48
+ .status-dot{width:7px;height:7px;border-radius:50%;background:var(--green);}
49
+ .head-meta{display:grid;grid-template-columns:repeat(4,1fr);gap:12px 24px;}
50
+ .head-meta .item{display:flex;flex-direction:column;gap:3px;}
51
+ .head-meta .k{font-family:var(--mono);font-size:9.5px;letter-spacing:.2em;color:var(--text-dim);text-transform:uppercase;}
52
+ .head-meta .v{font-family:var(--sans);font-size:13.5px;font-weight:700;color:var(--text);}
53
+
54
+ /* ---------- three honest badges ---------- */
55
+ .badges{display:grid;grid-template-columns:repeat(3,1fr);gap:12px;margin:18px 0 4px 0;}
56
+ .badge{background:var(--bg-card);border:1px solid var(--border-strong);border-radius:12px;padding:16px 20px;}
57
+ .badge .label{font-family:var(--mono);font-size:9.5px;letter-spacing:.2em;color:var(--text-dim);
58
+ text-transform:uppercase;margin-bottom:8px;}
59
+ .badge .value{font-family:var(--serif);font-style:italic;font-weight:700;font-size:26px;color:var(--text);}
60
+ .badge .sub{font-family:var(--mono);font-size:10px;letter-spacing:.1em;color:var(--text-mute);
61
+ text-transform:uppercase;margin-top:4px;}
62
+ .badge.green{border-color:rgba(31,209,96,.5);}
63
+ .badge.red{border-color:rgba(255,42,42,.5);}
64
+
65
+ /* ---------- exhibit (the museum lever) ---------- */
66
+ .exhibit{margin-top:18px;border:1px solid var(--border-strong);border-radius:14px;
67
+ background:var(--bg-card);padding:22px 24px 26px 24px;}
68
+ .exhibit-title{font-family:var(--serif);font-style:italic;font-weight:700;font-size:22px;margin:0 0 2px 0;}
69
+ .exhibit-lede{font-family:var(--mono);font-size:11px;letter-spacing:.1em;color:var(--text-mute);
70
+ text-transform:uppercase;margin-bottom:18px;}
71
+
72
+ .exhibit-grid{display:grid;grid-template-columns:1fr 300px;gap:22px;align-items:stretch;}
73
+ #chart{width:100%;height:430px;}
74
+ .readout{border:1px solid var(--border-strong);border-radius:12px;background:#050505;
75
+ padding:20px 22px;display:flex;flex-direction:column;gap:2px;transition:border-color 160ms ease;}
76
+ .readout.collapsed{border-color:rgba(255,42,42,.6);background:rgba(255,42,42,.04);}
77
+ .readout .ro-label{font-family:var(--mono);font-size:9.5px;letter-spacing:.2em;color:var(--text-dim);text-transform:uppercase;}
78
+ .readout .ro-name{font-family:var(--serif);font-style:italic;font-weight:700;font-size:30px;line-height:1.05;margin:4px 0 2px 0;}
79
+ .readout .ro-period{font-family:var(--mono);font-size:10.5px;letter-spacing:.08em;color:var(--text-mute);text-transform:uppercase;}
80
+ .ro-brier-label{font-family:var(--mono);font-size:9.5px;letter-spacing:.2em;color:var(--text-dim);
81
+ text-transform:uppercase;margin-top:20px;}
82
+ .ro-brier{font-family:var(--serif);font-style:italic;font-weight:700;font-size:56px;line-height:1;
83
+ color:var(--green);transition:color 160ms ease;}
84
+ .readout.collapsed .ro-brier{color:var(--red);}
85
+ .ro-tag{align-self:flex-start;margin-top:6px;padding:4px 10px;border-radius:999px;
86
+ font-family:var(--mono);font-size:9.5px;letter-spacing:.18em;text-transform:uppercase;
87
+ background:rgba(255,42,42,.1);border:1px solid rgba(255,42,42,.55);color:var(--red);
88
+ visibility:hidden;}
89
+ .readout.collapsed .ro-tag{visibility:visible;}
90
+ .ro-aux{margin-top:18px;border-top:1px solid var(--border);padding-top:14px;display:flex;flex-direction:column;gap:9px;}
91
+ .ro-aux .row{display:flex;align-items:baseline;justify-content:space-between;gap:10px;}
92
+ .ro-aux .rk{font-family:var(--mono);font-size:10px;letter-spacing:.06em;color:var(--text-dim);text-transform:uppercase;}
93
+ .ro-aux .rv{font-family:var(--mono);font-size:13px;font-weight:700;}
94
+ .rv.gold{color:var(--gold);} .rv.red{color:var(--red);} .rv.green{color:var(--green);} .rv.mute{color:var(--text-mute);}
95
+
96
+ /* ---------- slider (the lever) ---------- */
97
+ .lever{margin-top:24px;}
98
+ .lever-head{display:flex;align-items:baseline;justify-content:space-between;margin-bottom:14px;}
99
+ .lever-head .t{font-family:var(--mono);font-size:11px;letter-spacing:.18em;color:var(--text);text-transform:uppercase;}
100
+ .lever-head .h{font-family:var(--mono);font-size:10px;letter-spacing:.1em;color:var(--text-dim);text-transform:uppercase;}
101
+ .slider-area{position:relative;padding:0 4px;}
102
+ input[type=range].lever-input{
103
+ -webkit-appearance:none;appearance:none;width:100%;height:6px;border-radius:999px;margin:0;
104
+ background:linear-gradient(90deg,var(--green) 0%,var(--gold) 78%,var(--red) 100%);
105
+ outline:none;cursor:pointer;}
106
+ input[type=range].lever-input::-webkit-slider-thumb{
107
+ -webkit-appearance:none;appearance:none;width:26px;height:26px;border-radius:50%;
108
+ background:#0A0A0A;border:2px solid var(--gold);box-shadow:0 0 0 4px rgba(212,175,55,.18);
109
+ cursor:grab;margin-top:-10px;transition:border-color 120ms ease,box-shadow 120ms ease;}
110
+ input[type=range].lever-input::-webkit-slider-thumb:active{cursor:grabbing;}
111
+ input[type=range].lever-input::-moz-range-thumb{width:26px;height:26px;border-radius:50%;
112
+ background:#0A0A0A;border:2px solid var(--gold);box-shadow:0 0 0 4px rgba(212,175,55,.18);cursor:grab;}
113
+ input[type=range].lever-input::-moz-range-track{height:6px;border-radius:999px;background:transparent;}
114
+ input[type=range].lever-input.at-never::-webkit-slider-thumb{border-color:var(--red);box-shadow:0 0 0 4px rgba(255,42,42,.22);}
115
+ input[type=range].lever-input.at-never::-moz-range-thumb{border-color:var(--red);box-shadow:0 0 0 4px rgba(255,42,42,.22);}
116
+ input[type=range].lever-input:focus-visible::-webkit-slider-thumb{box-shadow:0 0 0 4px rgba(212,175,55,.20),0 0 0 7px var(--gold-hi);}
117
+ input[type=range].lever-input:focus-visible::-moz-range-thumb{box-shadow:0 0 0 4px rgba(212,175,55,.20),0 0 0 7px var(--gold-hi);}
118
+ .ticks{position:relative;height:10px;margin-top:9px;}
119
+ .tick{position:absolute;top:0;width:1px;height:6px;background:var(--border-strong);transform:translateX(-50%);}
120
+ .tick.on{background:var(--gold);height:9px;}
121
+ .ticklabels{position:relative;height:30px;margin-top:2px;}
122
+ .tlabel{position:absolute;top:0;font-family:var(--mono);font-size:9.5px;letter-spacing:.02em;
123
+ color:var(--text-dim);white-space:nowrap;line-height:1.15;text-align:center;cursor:pointer;
124
+ transition:color 120ms ease;}
125
+ .tlabel.on{color:var(--text);font-weight:700;}
126
+ .tlabel.never.on{color:var(--red);}
127
+ .lever-foot{margin-top:18px;display:flex;justify-content:space-between;
128
+ font-family:var(--mono);font-size:10px;letter-spacing:.12em;color:var(--text-dim);text-transform:uppercase;}
129
+
130
+ /* ---------- ablation strip ---------- */
131
+ .ablations{display:grid;grid-template-columns:repeat(3,1fr);gap:12px;margin:22px 0;}
132
+ .abl{background:var(--bg-card);border:1px solid var(--border-strong);border-radius:12px;padding:18px 20px;text-align:center;}
133
+ .abl .key{font-family:var(--mono);font-size:10px;letter-spacing:.16em;color:var(--text-dim);text-transform:uppercase;margin-bottom:8px;}
134
+ .abl .val{font-family:var(--serif);font-style:italic;font-weight:700;font-size:30px;color:var(--text);}
135
+ .abl .lbl{font-family:var(--mono);font-size:10px;letter-spacing:.1em;color:var(--text-mute);text-transform:uppercase;margin-top:4px;}
136
+ .abl.acc .val{color:var(--green);}
137
+
138
+ /* ---------- data table ---------- */
139
+ .table-wrap{background:var(--bg-card);border:1px solid var(--border-strong);border-radius:14px;padding:20px 22px;}
140
+ .table-cap{font-family:var(--mono);font-size:10px;letter-spacing:.18em;color:var(--text-dim);
141
+ text-transform:uppercase;margin-bottom:12px;}
142
+ .schedule-table{width:100%;border-collapse:collapse;font-family:var(--mono);font-size:13px;}
143
+ .schedule-table th{font-family:var(--mono);font-size:9.5px;letter-spacing:.18em;color:var(--text-dim);
144
+ text-transform:uppercase;text-align:right;padding:8px 12px;border-bottom:1px solid var(--border-strong);}
145
+ .schedule-table th:first-child{text-align:left;}
146
+ .schedule-table td{padding:9px 12px;border-bottom:1px solid var(--border);text-align:right;}
147
+ .schedule-table tr{cursor:pointer;}
148
+ .schedule-table tr.active{background:rgba(212,175,55,.07);}
149
+ .td-name{text-align:left!important;font-family:var(--serif);font-style:italic;font-weight:700;font-size:15px;color:var(--text);}
150
+ .td-mute{color:var(--text-mute);}
151
+ .td-c{color:var(--green);font-weight:700;} .td-b{color:var(--gold);font-weight:700;} .td-a{color:var(--red);font-weight:700;opacity:.78;}
152
+ .never-row{background:rgba(255,42,42,.06);}
153
+ .never-row .td-c{color:var(--red)!important;}
154
+
155
+ /* ---------- about ---------- */
156
+ .foot{margin-top:24px;padding:22px 24px;border:1px solid var(--border);border-radius:12px;background:var(--bg-card);}
157
+ .foot .ftitle{font-family:var(--mono);font-size:10px;letter-spacing:.2em;color:var(--text-dim);text-transform:uppercase;margin-bottom:10px;}
158
+ .foot .body{font-family:var(--sans);font-size:13px;color:var(--text-mute);line-height:1.62;}
159
+ .foot code{font-family:var(--mono);font-size:12px;color:var(--text);background:var(--bg-elev);
160
+ padding:1px 5px;border-radius:4px;border:1px solid var(--border);}
161
+ .attrib{margin-top:14px;padding-top:14px;border-top:1px solid var(--border);
162
+ font-family:var(--mono);font-size:11px;letter-spacing:.04em;color:var(--text-dim);line-height:1.7;}
163
+
164
+ @media(max-width:860px){
165
+ .exhibit-grid{grid-template-columns:1fr;}
166
+ .head-meta{grid-template-columns:repeat(2,1fr);}
167
+ .badges,.ablations{grid-template-columns:1fr;}
168
+ .head-top{flex-direction:column;}
169
+ .head-right{text-align:left;}
170
+ }
171
+ </style>
172
+ </head>
173
+ <body>
174
+ <main class="wrap">
175
+
176
+ <!-- HEADER -->
177
+ <header class="head">
178
+ <div class="head-top">
179
+ <div class="head-brand">
180
+ <div class="head-icon">
181
+ <svg width="28" height="28" viewBox="0 0 28 28" fill="none" aria-hidden="true">
182
+ <rect x="3.5" y="3.5" width="21" height="21" rx="4" stroke="#2A2A2A"/>
183
+ <line x1="14" y1="5.5" x2="14" y2="22.5" stroke="#D4AF37" stroke-width="1.6"/>
184
+ <circle cx="14" cy="9" r="1.3" fill="#1FD160"/>
185
+ <circle cx="14" cy="14" r="1.3" fill="#1FD160"/>
186
+ <circle cx="14" cy="19" r="1.3" fill="#1FD160"/>
187
+ </svg>
188
+ </div>
189
+ <div>
190
+ <div class="head-title">Thin Channel</div>
191
+ <div class="head-subtitle">How rarely external truth can arrive before calibration collapses</div>
192
+ </div>
193
+ </div>
194
+ <div class="head-right">
195
+ <span class="submitted-label">Submitted by</span>
196
+ <div class="submitted-name" id="author">—</div>
197
+ <div class="status-pill"><span class="status-dot"></span>Static demo · no live model</div>
198
+ </div>
199
+ </div>
200
+ <div class="head-meta">
201
+ <div class="item"><span class="k">Subject model</span><span class="v">Qwen2.5-3B-Instruct (frozen)</span></div>
202
+ <div class="item"><span class="k">Dataset</span><span class="v" id="m-dataset">ForecastBench</span></div>
203
+ <div class="item"><span class="k">Hold-out</span><span class="v" id="m-holdout">N = 330</span></div>
204
+ <div class="item"><span class="k">Calibration</span><span class="v">per-topic Beta · K = 10</span></div>
205
+ </div>
206
+ </header>
207
+
208
+ <!-- THREE HONEST BADGES -->
209
+ <section class="badges">
210
+ <div class="badge green">
211
+ <div class="label">Corridor — any finite schedule</div>
212
+ <div class="value" id="b-corridor">0.205–0.220</div>
213
+ <div class="sub">Brier on hold-out</div>
214
+ </div>
215
+ <div class="badge red">
216
+ <div class="label">Never — no signal</div>
217
+ <div class="value" id="b-never">0.298</div>
218
+ <div class="sub">collapses to raw 3B</div>
219
+ </div>
220
+ <div class="badge">
221
+ <div class="label">Cliff direction</div>
222
+ <div class="value">Binary</div>
223
+ <div class="sub">channel = volume, not frequency</div>
224
+ </div>
225
+ </section>
226
+
227
+ <!-- EXHIBIT: the museum lever -->
228
+ <section class="exhibit">
229
+ <div class="exhibit-title">How often does the system get to check reality?</div>
230
+ <div class="exhibit-lede">Move the lever from “every day” to “never” and watch the Brier curve</div>
231
+
232
+ <div class="exhibit-grid">
233
+ <div id="chart"></div>
234
+ <div class="readout" id="readout">
235
+ <div class="ro-label">Current check interval</div>
236
+ <div class="ro-name" id="ro-name">day</div>
237
+ <div class="ro-period" id="ro-period">every 1 day · 1316 outcomes seen</div>
238
+ <div class="ro-brier-label">Brier on hold-out · lower is better</div>
239
+ <div class="ro-brier" id="ro-brier">0.205</div>
240
+ <div class="ro-tag" id="ro-tag">Collapse — back to raw 3B</div>
241
+ <div class="ro-aux">
242
+ <div class="row"><span class="rk">(b) base-rate only</span><span class="rv gold" id="ro-b">0.210</span></div>
243
+ <div class="row"><span class="rk">(a) raw 3B only</span><span class="rv red" id="ro-a">0.298</span></div>
244
+ <div class="row"><span class="rk">improvement vs raw 3B</span><span class="rv" id="ro-delta">−0.093</span></div>
245
+ </div>
246
+ </div>
247
+ </div>
248
+
249
+ <div class="lever">
250
+ <div class="lever-head">
251
+ <span class="t">The lever — reveal schedule</span>
252
+ <span class="h">← more often &nbsp;·&nbsp; less often →</span>
253
+ </div>
254
+ <div class="slider-area">
255
+ <input id="slider" class="lever-input" type="range" min="0" max="10" step="1" value="0"
256
+ aria-label="How often the system checks reality" />
257
+ <div class="ticks" id="ticks"></div>
258
+ <div class="ticklabels" id="ticklabels"></div>
259
+ </div>
260
+ <div class="lever-foot">
261
+ <span>Daily contact</span>
262
+ <span>One annual debrief</span>
263
+ <span>Never</span>
264
+ </div>
265
+ </div>
266
+ </section>
267
+
268
+ <!-- ABLATION STRIP -->
269
+ <section class="ablations">
270
+ <div class="abl">
271
+ <div class="key">(a) raw 3B only</div>
272
+ <div class="val" id="abl-a">0.298</div>
273
+ <div class="lbl">useless as a forecaster</div>
274
+ </div>
275
+ <div class="abl acc">
276
+ <div class="key">(b) topic base-rate only</div>
277
+ <div class="val" id="abl-b">0.189</div>
278
+ <div class="lbl">no model needed</div>
279
+ </div>
280
+ <div class="abl acc">
281
+ <div class="key">(c) mix (a + b)</div>
282
+ <div class="val" id="abl-c">0.189</div>
283
+ <div class="lbl">c − b = <span id="abl-d">−0.0001</span></div>
284
+ </div>
285
+ </section>
286
+
287
+ <!-- DATA TABLE -->
288
+ <section class="table-wrap">
289
+ <div class="table-cap">All 11 schedules — verbatim from the run (click a row to move the lever)</div>
290
+ <table class="schedule-table">
291
+ <thead><tr>
292
+ <th>Frequency</th><th>Period</th><th>Revealed</th>
293
+ <th>(c) mix</th><th>(b) base only</th><th>(a) raw 3B</th>
294
+ </tr></thead>
295
+ <tbody id="tbody"></tbody>
296
+ </table>
297
+ </section>
298
+
299
+ <!-- ABOUT -->
300
+ <footer class="foot">
301
+ <div class="ftitle">About this demo</div>
302
+ <div class="body">
303
+ 1,646 resolved binary forecasting questions are streamed through a simulated calendar:
304
+ a frozen Qwen2.5-3B-Instruct emits a raw <i>P(YES)</i> at each question’s freeze date,
305
+ the true outcome is hidden and revealed only on a schedule, and a per-topic Beta
306
+ calibration (9 topics, K = 10) consumes each revealed outcome. The same stream is run
307
+ independently through 11 reveal schedules, from “every day” to “never”. The metric is
308
+ Brier score on the held-out last 20% (N = 330); lower is better.
309
+ The result is a binary cliff: every finite schedule holds the corridor at
310
+ <b id="t-corridor">0.205–0.220</b>, while zero contact collapses to
311
+ <b id="t-never">0.298</b> — bit-for-bit the raw 3B level.
312
+ <br/><br/>
313
+ Read the honest caveat alongside it: the raw 3B is useless as a forecaster on its own
314
+ (Brier <span id="t-wide">0.354</span> on a wide baseline, worse than “always NO”). The (c) ≈ (b) ablation shows
315
+ the entire calibration gain comes from picking up <b>per-topic base rates</b>, not from
316
+ the model learning to predict the future. What the lever measures is the rate at which
317
+ unforgeable signal accumulates — not forecasting skill.
318
+ <br/><br/>
319
+ No live model runs in this Space. Every number, the curve and the table are verbatim
320
+ outputs of the original experimental run, bundled into <code>data.json</code>. Full code,
321
+ raw per-schedule results and methodology:
322
+ <a id="repo-link" href="https://github.com/SergheiBrinza/thin-channel" target="_blank" rel="noopener">github.com/SergheiBrinza/thin-channel</a>.
323
+ </div>
324
+ <div class="attrib" id="attrib">
325
+ Dataset: ForecastBench (Karger, Bastani, Yueh-Han, Jacobs, Halawi, Zhang &amp; Tetlock,
326
+ <i>ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities</i>, ICLR 2025,
327
+ arXiv:2409.19839), licensed CC BY-SA 4.0 — source:
328
+ <a href="https://huggingface.co/datasets/forecastingresearch/forecastbench-datasets" target="_blank" rel="noopener">forecastingresearch/forecastbench-datasets</a>.
329
+ No raw market questions or per-question outcomes are redistributed here — only aggregate
330
+ Brier results. Demo code: MIT.
331
+ </div>
332
+ </footer>
333
+
334
+ </main>
335
+
336
+ <script>
337
+ const COL={bg:'#000000',card:'#0A0A0A',border:'#1F1F1F',borderS:'#2A2A2A',
338
+ text:'#FFFFFF',mute:'#A8A8A8',dim:'#6B6B6B',gold:'#D4AF37',red:'#FF2A2A',green:'#1FD160'};
339
+ const f3=v=>v.toFixed(3);
340
+ const signed=v=>(v>0?'+':v<0?'−':'')+Math.abs(v).toFixed(3);
341
+ const signed4=v=>(v>0?'+':v<0?'−':'')+Math.abs(v).toFixed(4);
342
+
343
+ let S=[], NAMES=[], FINAL=[], BASE=[], RAW=[], REVEAL=[], FLOOR=0, CORR_MIN=0, CORR_MAX=0, cur=0;
344
+
345
+ const PLOT_MARGIN={l:54,r:18,t:14,b:30};
346
+
347
+ fetch('data.json').then(r=>r.json()).then(D=>{
348
+ const meta=D.meta||{};
349
+ S=D.schedules;
350
+ NAMES=S.map(s=>s.name);
351
+ FINAL=S.map(s=>s.holdout_brier_final);
352
+ BASE=S.map(s=>s.holdout_brier_base_only);
353
+ RAW=S.map(s=>s.holdout_brier_raw);
354
+ REVEAL=S.map(s=>Math.round(s.total_revealed));
355
+ const finite=S.filter(s=>s.period_days!==null);
356
+ CORR_MIN=Math.min(...finite.map(s=>s.holdout_brier_final));
357
+ CORR_MAX=Math.max(...finite.map(s=>s.holdout_brier_final));
358
+ FLOOR=Math.min(...finite.map(s=>s.holdout_brier_base_only));
359
+
360
+ // header / badges / about — all driven by data
361
+ document.getElementById('author').textContent=meta.author||'—';
362
+ document.getElementById('m-dataset').textContent=meta.dataset_short||'ForecastBench';
363
+ document.getElementById('m-holdout').textContent='N = '+(meta.n_holdout||330);
364
+ const corrTxt=f3(CORR_MIN)+'–'+f3(CORR_MAX);
365
+ document.getElementById('b-corridor').textContent=corrTxt;
366
+ document.getElementById('t-corridor').textContent=corrTxt;
367
+ const neverV=S.find(s=>s.period_days===null).holdout_brier_final;
368
+ document.getElementById('b-never').textContent=f3(neverV);
369
+ document.getElementById('t-never').textContent=f3(neverV);
370
+ const repo=meta.repo||'https://github.com/SergheiBrinza/thin-channel';
371
+ const rl=document.getElementById('repo-link'); rl.href=repo;
372
+ rl.textContent=repo.replace(/^https?:\/\//,'');
373
+ const abl=D.ablations||{};
374
+ document.getElementById('abl-a').textContent=f3(abl.brier_a_raw);
375
+ document.getElementById('abl-b').textContent=f3(abl.brier_b_base_only);
376
+ document.getElementById('abl-c').textContent=f3(abl.brier_c_mix);
377
+ document.getElementById('abl-d').textContent=signed4(abl.delta_c_vs_b);
378
+ if(D.wide_baseline_overall&&typeof D.wide_baseline_overall.brier==='number'){
379
+ const tw=document.getElementById('t-wide'); if(tw) tw.textContent=f3(D.wide_baseline_overall.brier);
380
+ }
381
+
382
+ buildTicks();
383
+ buildTable();
384
+ drawChart();
385
+
386
+ // optional deep-link / screenshot preset: ?i=10
387
+ const q=new URLSearchParams(location.search);
388
+ const i=parseInt(q.get('i'),10);
389
+ const start=(Number.isFinite(i)&&i>=0&&i<=10)?i:0;
390
+ document.getElementById('slider').value=start;
391
+ document.getElementById('slider').addEventListener('input',e=>setPos(+e.target.value));
392
+ setPos(start);
393
+ }).catch(err=>{
394
+ console.error('thin-channel: could not load data.json —', err);
395
+ const ex=document.querySelector('.exhibit');
396
+ if(ex){
397
+ const b=document.createElement('div');
398
+ b.style.cssText='border:1px solid rgba(255,42,42,.5);background:rgba(255,42,42,.06);'+
399
+ 'color:#FF2A2A;font-family:var(--mono);font-size:12px;letter-spacing:.04em;'+
400
+ 'padding:14px 16px;border-radius:10px;margin:4px 0 16px 0;line-height:1.5;';
401
+ b.textContent='Data unavailable — could not load data.json ('+err+'). '+
402
+ 'This static demo needs data.json served alongside the page.';
403
+ ex.insertBefore(b,ex.firstChild);
404
+ }
405
+ });
406
+
407
+ function buildTicks(){
408
+ const T=document.getElementById('ticks'), L=document.getElementById('ticklabels');
409
+ T.innerHTML=''; L.innerHTML='';
410
+ NAMES.forEach((nm,i)=>{
411
+ const pct=i/10*100;
412
+ const t=document.createElement('div'); t.className='tick'; t.dataset.i=i;
413
+ t.style.left=pct+'%'; T.appendChild(t);
414
+ const l=document.createElement('div'); l.className='tlabel'+(nm==='never'?' never':'');
415
+ l.dataset.i=i; l.textContent=nm;
416
+ l.style.left=pct+'%';
417
+ if(i===0){l.style.transform='translateX(0)';l.style.textAlign='left';}
418
+ else if(i===NAMES.length-1){l.style.transform='translateX(-100%)';l.style.textAlign='right';}
419
+ else{l.style.transform='translateX(-50%)';}
420
+ l.addEventListener('click',()=>{document.getElementById('slider').value=i;setPos(i);});
421
+ L.appendChild(l);
422
+ });
423
+ }
424
+
425
+ function buildTable(){
426
+ const tb=document.getElementById('tbody'); tb.innerHTML='';
427
+ S.forEach((s,i)=>{
428
+ const tr=document.createElement('tr');
429
+ tr.className=(s.period_days===null?'never-row':'')+ (i===cur?' active':'');
430
+ tr.dataset.i=i;
431
+ const period=s.period_days===null?'∞':s.period_days+'d';
432
+ tr.innerHTML='<td class="td-name">'+s.name+'</td>'+
433
+ '<td class="td-mute">'+period+'</td>'+
434
+ '<td class="td-mute">'+Math.round(s.total_revealed)+'</td>'+
435
+ '<td class="td-c">'+f3(s.holdout_brier_final)+'</td>'+
436
+ '<td class="td-b">'+f3(s.holdout_brier_base_only)+'</td>'+
437
+ '<td class="td-a">'+f3(s.holdout_brier_raw)+'</td>';
438
+ tr.addEventListener('click',()=>{document.getElementById('slider').value=i;setPos(i);});
439
+ tb.appendChild(tr);
440
+ });
441
+ }
442
+
443
+ const X=[...Array(11).keys()];
444
+ function drawChart(){
445
+ const axisFont={family:"'JetBrains Mono',monospace",size:11,color:COL.mute};
446
+ const traces=[
447
+ // corridor band edges drawn via shapes; lines below
448
+ {x:X,y:RAW,name:'(a) raw 3B',mode:'lines',line:{color:COL.red,width:1.6,dash:'dash',shape:'spline'},
449
+ opacity:.55,hovertemplate:'(a) raw 3B<br>%{y:.3f}<extra></extra>'},
450
+ {x:X,y:BASE,name:'(b) base rate',mode:'lines+markers',line:{color:COL.gold,width:2,shape:'spline'},
451
+ marker:{size:5,color:COL.gold},hovertemplate:'(b) base rate<br>%{y:.3f}<extra></extra>'},
452
+ {x:X,y:FINAL,name:'(c) mix (ours)',mode:'lines+markers',line:{color:COL.green,width:3,shape:'spline'},
453
+ marker:{size:6,color:COL.green},hovertemplate:'(c) mix<br>%{y:.3f}<extra></extra>'},
454
+ // moving highlight point
455
+ {x:[0],y:[FINAL[0]],name:'',mode:'markers',showlegend:false,hoverinfo:'skip',
456
+ marker:{size:18,color:COL.green,line:{color:'#fff',width:2.5}}}
457
+ ];
458
+ const layout=baseLayout();
459
+ Plotly.newPlot('chart',traces,layout,{displayModeBar:false,responsive:true});
460
+ }
461
+
462
+ function baseLayout(){
463
+ return {
464
+ paper_bgcolor:COL.card, plot_bgcolor:COL.card,
465
+ margin:PLOT_MARGIN, height:430,
466
+ font:{family:"'JetBrains Mono',monospace",color:COL.mute},
467
+ showlegend:true,
468
+ legend:{orientation:'h',y:1.12,x:0,font:{size:11,color:COL.mute},bgcolor:'rgba(0,0,0,0)'},
469
+ hoverlabel:{bgcolor:'#141414',bordercolor:COL.borderS,font:{family:"'JetBrains Mono',monospace",color:COL.text,size:12}},
470
+ xaxis:{tickmode:'array',tickvals:X,ticktext:NAMES,tickangle:-35,
471
+ tickfont:{family:"'JetBrains Mono',monospace",size:9.5,color:COL.dim},
472
+ gridcolor:COL.border,zeroline:false,range:[-0.4,10.4],fixedrange:true},
473
+ yaxis:{title:{text:'Brier (lower = better)',font:{size:11,color:COL.mute}},
474
+ tickfont:{family:"'JetBrains Mono',monospace",size:10,color:COL.dim},
475
+ gridcolor:COL.border,zeroline:false,range:[0.196,0.312],fixedrange:true},
476
+ shapes:shapesFor(cur),
477
+ annotations:annotationsFor(cur)
478
+ };
479
+ }
480
+
481
+ function shapesFor(i){
482
+ const isNever=(S[i]&&S[i].period_days===null);
483
+ return [
484
+ // corridor band 0.205–0.220
485
+ {type:'rect',xref:'paper',x0:0,x1:1,yref:'y',y0:CORR_MIN,y1:CORR_MAX,
486
+ fillcolor:'rgba(31,209,96,0.06)',line:{width:0},layer:'below'},
487
+ // base-only floor reference line
488
+ {type:'line',xref:'paper',x0:0,x1:1,yref:'y',y0:FLOOR,y1:FLOOR,
489
+ line:{color:COL.gold,width:1,dash:'dot'},layer:'below'},
490
+ // vertical guide connecting the lever to the point
491
+ {type:'line',xref:'x',x0:i,x1:i,yref:'paper',y0:0,y1:1,
492
+ line:{color:isNever?'rgba(255,42,42,0.5)':'rgba(255,255,255,0.16)',width:1},layer:'below'}
493
+ ];
494
+ }
495
+
496
+ function annotationsFor(i){
497
+ const a=[{
498
+ xref:'paper',yref:'y',x:0.012,y:FLOOR,xanchor:'left',yanchor:'bottom',
499
+ text:'base-only floor ≈ '+FLOOR.toFixed(2),showarrow:false,
500
+ font:{family:"'JetBrains Mono',monospace",size:9.5,color:COL.gold}
501
+ }];
502
+ if(S[i]&&S[i].period_days===null){
503
+ a.push({xref:'x',yref:'y',x:10,y:FINAL[10],ax:-46,ay:0,axref:'pixel',ayref:'pixel',
504
+ text:'collapse',showarrow:true,arrowhead:2,arrowcolor:COL.red,arrowwidth:1.4,
505
+ font:{family:"'JetBrains Mono',monospace",size:11,color:COL.red},
506
+ xanchor:'right',yanchor:'middle'});
507
+ }
508
+ return a;
509
+ }
510
+
511
+ function setPos(i){
512
+ cur=i;
513
+ const s=S[i], isNever=(s.period_days===null);
514
+ // chart: move highlight point, color, guide line, collapse annotation
515
+ Plotly.restyle('chart',{x:[[i]],y:[[FINAL[i]]],'marker.color':isNever?COL.red:COL.green},[3]);
516
+ Plotly.relayout('chart',{shapes:shapesFor(i),annotations:annotationsFor(i)});
517
+ // readout
518
+ const ro=document.getElementById('readout');
519
+ ro.classList.toggle('collapsed',isNever);
520
+ document.getElementById('ro-name').textContent=s.name;
521
+ document.getElementById('ro-period').textContent =
522
+ (isNever?'no reveals · 0 outcomes seen'
523
+ :'every '+s.period_days+' day'+(s.period_days===1?'':'s')+' · '+REVEAL[i]+' outcomes seen');
524
+ document.getElementById('ro-brier').textContent=f3(FINAL[i]);
525
+ document.getElementById('ro-b').textContent=f3(BASE[i]);
526
+ document.getElementById('ro-a').textContent=f3(RAW[i]);
527
+ const delta=FINAL[i]-RAW[i];
528
+ const dEl=document.getElementById('ro-delta');
529
+ dEl.textContent = delta<-0.0005 ? signed(delta)+' Brier' : '0.000 — none';
530
+ dEl.className='rv '+(delta<-0.0005?'green':'red');
531
+ // slider thumb tint
532
+ const sl=document.getElementById('slider');
533
+ sl.classList.toggle('at-never',isNever);
534
+ sl.setAttribute('aria-valuetext', s.name+' — Brier '+f3(FINAL[i]));
535
+ // ticks + labels active state
536
+ document.querySelectorAll('.tick').forEach(t=>t.classList.toggle('on',+t.dataset.i===i));
537
+ document.querySelectorAll('.tlabel').forEach(l=>l.classList.toggle('on',+l.dataset.i===i));
538
+ // table active row
539
+ document.querySelectorAll('#tbody tr').forEach(tr=>tr.classList.toggle('active',+tr.dataset.i===i));
540
+ }
541
+ </script>
542
+ </body>
543
+ </html>
requirements.txt DELETED
@@ -1,2 +0,0 @@
1
- gradio>=5.0,<6
2
- huggingface_hub<1.0