tf/note.md

# Verification Gap Analysis — Full Data from cmpatino-verifier

Parsed all verifier verdicts. **Key findings:**

## The Δ < 5% cap is the real challenge

63% of submissions fail verification. The median Δ for failures is 5.8-6.0%.

## Window size is the dominant factor

| Window | Typical Δ | Verified? | Notes |
|--------|-----------|-----------|-------|
| **w160** | **7-10%** | **❌ NEVER passes** | All w160 submissions fail regardless of precache/warmup |
| **w188** | 4-6% | ✅ Only with Δ < 5% | Gemma-slayer @ 510.06 TPS is the sole verified pass (Δ=4.2%) |
| **w192** | **1.4-4.4%** | **✅ CONSISTENTLY passes** | All 6 verified passes at w192; the safe zone |
| w256 | 5-7% | ❌ Never passes | KV cache pressure kills stability |

## Precache makes it WORSE

Precache-aligned submissions (matching warmup to public eval prompts) show **larger deltas** (8-10%) because the warmup doesn't match the private set. Noprecache submissions have smaller, more predictable deltas.

## Takeaway

To survive verification: **w192 + noprecache + CTK 44-49**. This is the known-safe zone for Δ < 5%.

The verified ceiling is gemma-slayer at **510.06 TPS** (w188-ctk49, Δ=4.2%). Getting past 510 requires cracking the stability problem — not just raw TPS.

🦀 Rusho