# Audit trail — from the original kit to this one

This file documents what was removed, changed, or rewritten when the
project went from `tilelli-kit-2026-05-23-audited.zip` (the original
shipping kit, SHA `00af96d8…`) to this humble public repo.

## Headline

The original kit shipped 6 forward-looking claims ("emergent
metacognition", "3/3 seeds beat vanilla", "matched at parameter count",
etc.). This kit ships **3 verifiable positives + 3 verified negatives**,
and only the v4 deployed ckpt.

## Removed

| Artifact | Why |
|---|---|
| `BENCHMARKS.md` (original) | Headline overstated: "3/3 seeds beat vanilla" rested on heterogeneous `best_val` definition + asymmetric `eval_every`. Replaced by `results/claim_01_benchmark.md` (preliminary 1-seed directional). |
| `HOW_TILELLI_WORKS.html` | 17-scene marketing-style explainer. Replaced by this README + the audit trail. |
| `PAPER.md` §4.2 "emergent metacognition" | Empirically disproven, see `results/claim_02_metacog.md`. |
| Ckpts: `tilelli_chat_v5_metacog_cpu.pt`, `tilelli_chat_v6_metacog_chat_step100.pt`, `tilelli_chat_v7_metacog_v2.pt`, `tilelli_chat_v4splice_v7abstain.pt` | All failed their respective deploy gates. Available on request for negative-result replication. |
| `runs/` (28 of 30 sub-dirs) | Historical engineering churn. Only the 3 audit-relevant ones (`metacog_2026-05-23`, `metacog_v7_2026-05-23`, `metacog_v4v7splice_2026-05-23`) are referenced by claims here. |
| `swap_weights.sh` | Deprecated per 2026-05-20 memory; could cause prod regression. |

## Rewritten

| File | Change |
|---|---|
| `README.md` | Karpathy-style honest framing. Every claim links to a script. Negative results first-class. |
| `results/claim_01_benchmark.md` | Inherits the audited `BENCHMARKS.md` (21:39 2026-05-23) honest version. |
| `results/claim_02_metacog.md` | New, documents the DISPROVEN router-entropy claim + the v7 sub-result + the splice failure. |
| `pyproject.toml` | Trimmed to torch + numpy only. |

## Preserved as-is

| File | Notes |
|---|---|
| `src/tilelli/` (the package) | The actual model code. Unchanged from the kit version. |
| `checkpoints/tilelli_chat_v4.pt` | The deployed ckpt. SHA-pinned in `results/claim_01_benchmark.md`. |
| `prompts/probe_210.jsonl` | The audited eval set. |
| `LICENSE` | Apache 2.0. |

## What replication looks like

If you want to reproduce the negative results (not just the positives),
the original kit + the metacog run dirs (v5 / v6 / v7 / v8a / v8b ckpts
and the full probe JSONLs) are preserved in the lineage's working repo —
available on request via `hello@tilelli.tech`.

## What "humble" means here

Three rules:

1. **The README is the contract.** Every numerical claim has a script
   that produces it from a bundled artifact, and the script exits
   non-zero if it doesn't.
2. **Negative results are first-class.** A finding of "this doesn't
   work, and here's exactly how we know" is as worth shipping as a
   positive result of equal effort.
3. **One ckpt, one story.** Don't ship five ckpts and let the user
   guess which one corresponds to the headline. Ship the one that
   anchors the claims; document the others as audit trail.

This is the Karpathy-style framing of `nanoGPT` and `minBPE` adapted to a
small routed LM with an audited claim trail.