add README
Browse files
README.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# exp017 -- Where in the stack does header-bind need to live?
|
| 6 |
+
|
| 7 |
+
Public mirror of one experiment report from a private research repo.
|
| 8 |
+
|
| 9 |
+
[View report.pdf](report.pdf)
|
| 10 |
+
|
| 11 |
+
## Summary
|
| 12 |
+
|
| 13 |
+
Tests whether header_bind's WTQ/TableBench accuracy lift can be attributed to a specific 4-layer window of the Llama-3.2-1B stack. Three variants train the bias restricted to disjoint windows: early4 (layers 0-3), mid4 (4-7), late4 (12-15). The exp011 v2 gamma heatmap put strongest activity in mid-stack -- so the pre-registered hypothesis ordered mid4 > early4 > late4.
|
| 14 |
+
|
| 15 |
+
**Verdict: hypothesis FALSIFIED.** mid4 is paired-null (+0.12pp, McNemar p=1.00); late4 alone reproduces ~85% of the full-stack +1.80pp lift (+1.56pp, p=0.092 at n=835). Multi-hop FactChecking +12pp in ALL three windows (significant in early4 p=0.03) -- location-agnostic. mid4 produces the only significant NEGATIVE anywhere (DescriptiveAnalysis -15.7pp, p=0.01). Magnitude != effectiveness: late4 learned the smallest gamma values yet won.
|
| 16 |
+
|
| 17 |
+
Base: Llama-3.2-1B + LoRA. Train: TableInstruct DP subset. Eval: TableBench DP test, Claude Sonnet LLM-judge.
|