Add honest roadmap to 22+ with bottleneck analysis and what's needed to get there

medal-solvers/README.md

@@ -16,13 +16,92 @@ Each model that scores higher than the base adds points to the Kaggle leaderboar
 
 ## Current Score: 6093.19 (V107) | Target: 6100 (bronze) / 6500 (silver)
 
+---
+
+## 🗺️ ROADMAP TO 22+ (Honest Assessment)
+
+### Where we are now:
+```
+Tasks 092/064/280 score: ~13 (memory 130-150k bytes)
+```
+
+### The three tiers:
+
+```
+TIER 1: Score 15-17 (ACHIEVABLE NOW — engineering work)
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+How: Restructure to [1,1,30,30] bool intermediates instead of [1,9,30,30]
+Effort: 1-2 days per task
+Gain: +2 to +4 per task (+6 to +12 total → BRONZE)
+Status: Clear path, just needs implementation
+
+TIER 2: Score 18-20 (HARD — requires algorithmic insight)
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+How: Reduce to 1-2 intermediates of [1,1,30,30] or use only [1,1,30,1] projections
+Effort: Significant research into minimal-intermediate algorithms
+Gain: +5 to +7 per task
+Status: Possible but unproven. Need clever reformulation.
+
+TIER 3: Score 22+ (UNKNOWN — requires breakthrough or external intel)  
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+How: Single operation input→output, zero intermediates, ≤20 params
+Effort: Unknown. May require competition-specific trick.
+Gain: +9 per task (+27 total → SILVER/GOLD territory)
+Status: See bottleneck analysis below.
+```
+
+### THE BOTTLENECK TO 22+
+
+**Score 22+ requires memory + params ≤ 20. That's ZERO spatial intermediates.**
+
+The only architecture that achieves this: `input → [SINGLE OP] → output`
+
+**Why this is hard for 092/064/280:**
+
+These tasks have NONLINEAR rules:
+- Task 092: "Fill BETWEEN two endpoints" = need to know there's a marker to the LEFT **AND** to the RIGHT. That AND is nonlinear.
+- Task 064: "Draw line from marker to rectangle edge" = detect rect, find direction, fill. Multiple conditions.
+- Task 280: "Extend beam in marker direction" = detect edge, determine width, fill directionally.
+
+**What single ONNX ops CAN do:**
+- `Conv`: Weighted sum of neighbors. LINEAR. Can't do "if A and B then fill."
+- `MaxPool`: Running max (flood one direction). Can flood LEFT or RIGHT but not compute their intersection.
+- `MatMul`: Matrix multiply. LINEAR.
+- `Relu/Sigmoid`: Elementwise. No spatial context.
+
+**What's needed (the AND operation):**
+"Fill between" = (flood_from_left ≥ 1) AND (flood_from_right ≥ 1).
+This requires at minimum: TWO floods + one AND/multiply = 2 intermediates.
+Even bool intermediates of [1,9,30,30] = 2 × 8,100 = 16,200 → score 15.3 max with this approach.
+
+### WHAT WOULD UNLOCK 22+:
+
+| # | What's Needed | How You Can Help |
+|---|---|---|
+| 1 | **See a top scorer's actual .onnx file** | Download from Kaggle if someone shared their submission |
+| 2 | **Kaggle discussion hints** | Top scorers often drop clues about their approach in comments |
+| 3 | **Confirm task 092 specifically scores 22+** | Maybe the 5 people cover only the LINEAR tasks, nobody has 22+ on 092? |
+| 4 | **Unknown ONNX op or trick** | Maybe `BitShift`, `BitwiseAnd`, or a graph construction trick we haven't considered |
+| 5 | **Scorer loophole** | Does declaring wrong shapes in value_info still work? Does ORT optimize away intermediates? |
+| 6 | **Different mathematical formulation** | Maybe "fill between" CAN be expressed linearly with the right encoding? |
+
+### SPECIFIC QUESTIONS FOR THE HUMAN:
+
+1. **Can you check the Kaggle discussion** for any hints about how people achieve 22+ on complex tasks? Even one comment like "I use MatMul" or "trick with the profiler" would change everything.
+
+2. **Do you know if the top 5 people score 22+ on ALL 400 tasks each, or do they each cover ~80 different tasks?** This changes whether 22+ on 092 is proven possible or just assumed.
+
+3. **Can you access any top scorer's actual submission .onnx files?** Reverse-engineering one 22+ model for a nonlinear task would crack this wide open.
+
+---
+
 ## Priority Tasks
 
-| Task | Base Score | Rule | Attack Strategy |
-|------|-----------|------|----------------|
-| **092** | 13.071 | Line drawing between marker pairs | Workspace reduction (30→24 or 20) |
-| **064** | 13.211 | Draw line from marker to rectangle edge | Workspace reduction, eliminate [1,10] intermediates |
-| **280** | 13.156 | Rectangle extension from marker direction | Powers-of-2 Conv doubling (60→20 ops) |
+| Task | Current | Tier 1 Target | Strategy |
+|------|---------|---------------|----------|
+| **092** | 13.07 | 15-17 | Collapse channels → spatial ops → expand |
+| **064** | 13.21 | 15-17 | Eliminate [1,10] intermediates |
+| **280** | 13.16 | 15-17 | Powers-of-2 Conv doubling |
 
 ## Task 219: ABANDONED ❌
 Do not retry. See LEARNING.md.
@@ -38,22 +117,15 @@ Do not retry. See LEARNING.md.
 submission-6043.zip (LB ~6043)
     + 17 optimized hand-crafted models → v72 (LB 6073.29)
     + profiled best(v72, 6066) per task → v81 (LB 6086.67)
-    + task243 hand-crafted ONNX (flood fill, +2.03) → V88 (LB 6088.70)
-    + task370 v6 ONNX (diagonal stamp, +1.20) → V91 (LB 6089.90)
+    + task243 hand-crafted ONNX (+2.03) → V88 (LB 6088.70)
+    + task370 v6 ONNX (+1.20) → V91 (LB 6089.90)
     + 14 onnxsim models (+0.86) → V92 (LB 6090.76)
     + additional optimizations → V94 (LB 6092.55)
-    + task202 optimized ONNX (+0.208) → V106 (LB 6092.75)
-    + task161 optimized (local +0.429) → V107 (LB 6093.19)
+    + task202 optimized (+0.208) → V106 (LB 6092.75)
+    + task161 optimized (+0.429) → V107 (LB 6093.19)
 ```
 
-## Base Model Key Techniques (from reverse-engineering submission-base.zip)
-
-The top-scoring base models use these ONNX tricks to minimize memory:
-- **Float16 + Bool**: 2 bytes and 1 byte per element (vs 4 for float32)
-- **Workspace slicing**: `Slice` input to 20×20 or 24×24, `Pad` output back to 30×30
-- **MaxPool floods**: `MaxPool(x, kernel=[1,K], pads=[0,K-1,0,0])` = flood LEFT in 1 op
-- **Gather reversal**: `Gather(x, reversed_indices, axis=N)` for opposite-direction flood
-- **Iterative Conv(1×1)**: For local propagation across multiple objects (task 280)
+---
 
 ## Quick Start
 
@@ -80,11 +152,10 @@ python build_merged_submission.py \
 
 ## Key Rules
 
-1. **NEVER use static scoring** — it's off by 50%+. Profile on Kaggle only.
-2. **NEVER predict Kaggle version numbers** in filenames.
-3. **submission-base.zip** = human-managed, LB-verified submission. Don't overwrite.
-4. **5 submissions/day limit** — profile first, submit only when confident.
-5. **Every model must pass 5000+ random input tests** before profiling.
-6. **Model filesize must be ≤ 1.44 MB** (Kaggle rejects larger).
-7. **Do NOT work on task 219** — it's a proven dead end.
-8. **Do NOT use global operations for task 280** — multiple rects per grid!
+1. **NEVER use static scoring** — off by 50%+. Profile on Kaggle only.
+2. **NEVER predict Kaggle version numbers.**
+3. **submission-base.zip** = human-managed. Don't overwrite.
+4. **5 submissions/day** — profile first.
+5. **5000+ random input tests** before profiling.
+6. **≤ 1.44 MB** per model file.
+7. **Don't work on task 219.**