README.md

metadata

title: 3 · Thin Channel
emoji: 🔁
colorFrom: indigo
colorTo: purple
sdk: static
app_file: index.html
pinned: true
license: mit
short_description: Rare external truth before calibration collapses

Thin Channel — interactive demo

Interactive visualization of Experiment 6 (exp6-forecast-calibration) of the Second Loop project.

This Space loads no model. Everything is a static page driven by data.json — the verbatim aggregate output of the live experimental run.

The exhibit

Move the lever — how often the system gets to check reality — across its 11 settings, from every day to never. The Brier curve redraws live and the highlighted point tracks your setting. Drive the lever all the way to never and the point jumps off the corridor up to 0.298 — the system collapses bit-for-bit back to the raw 3B baseline. That is the whole finding, in one gesture.

What the numbers say

• 11 reveal schedules tested (day → year + never).
• Corridor of 0.205 – 0.220 Brier on hold-out for any finite schedule.
• 0.298 on never — collapses to the raw 3B baseline.
• The cliff is binary: channel thickness is total signal volume across the system's lifetime, not the frequency at which it arrives.
• Honest caveat (shown in the page): the raw 3B is useless as a forecaster on its own; the (c) ≈ (b) ablation shows the entire calibration gain comes from per-topic base rates, not from the model learning to predict.

Data and attribution

The forecasting questions come from ForecastBench (Karger, Bastani, Yueh-Han, Jacobs, Halawi, Zhang & Tetlock, ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities, ICLR 2025, arXiv:2409.19839), licensed CC BY-SA 4.0 — forecastingresearch/forecastbench-datasets. No raw market questions or per-question outcomes are redistributed here — only aggregate Brier results bundled in data.json. Demo code: MIT.

Source code, raw per-schedule JSON results, and methodology document: https://github.com/SergheiBrinza/thin-channel