Add results: comparison_table.md

results/comparison_table.md

@@ -0,0 +1,38 @@
+# gnarp-m2 Comparison Table
+
+## Cross-Domain Transfer Benchmark v2 (36 tasks, heuristic scoring)
+
+| Model | Judge Mean | vs Base | Avg Response (chars) | Avg Latency (s) |
+|-------|-----------|---------|---------------------|-----------------|
+| **gnarp-m2** | **0.7839** | **+14.1%** | 1,108 | 4.9 |
+| base (SmolLM2-360M-Instruct) | 0.6871 | — | 1,489 | 6.9 |
+
+## Prior gnarp-m1 Lineage (heldout benchmark, qwen3:8b judge, 36 tasks)
+
+| Model | Transfer | vs Base | Loss | Perplexity | Refusal | Train Data |
+|-------|---------|---------|------|-----------|---------|-----------|
+| base | 0.709 | — | 1.625 | 5.08 | 0.130 | — |
+| v1 | 0.218 | -69.2% | 1.850 | 6.36 | 0.385 | ~2,152 |
+| v2 | 0.713 | +0.6% | 1.800 | 6.05 | 0.340 | ~2,152 |
+| v3 | 0.561 | -20.9% | 1.790 | 5.99 | 0.400 | ~2,152 |
+| **m2** | **0.7839*** | **+14.1%*** | 2.509† | — | — | **74,395** |
+
+\* m2 scored on transfer_benchmark_v2 (heuristic-only), not the qwen3:8b-judged heldout benchmark.
+† m2 eval loss is from QLoRA training held-out split, not the same heldout eval as v1–v3.
+
+## Training Evolution
+
+| Version | Corpus Size | Epochs | Method | Key Change |
+|---------|-----------|--------|--------|-----------|
+| v1 | ~2,152 | 2 | QLoRA r=8 | First fine-tune, data loss incident |
+| v2 | ~2,152 | 2 | QLoRA r=8 | Fixed tag scaffolding poison |
+| v3 | ~2,152 | 2 | QLoRA r=8 | Positive-only targets |
+| **m2** | **74,395** | **1** | **QLoRA r=16** | **35x more data, single epoch** |
+
+## Key Takeaways
+
+1. **Data scale matters more than epochs.** m2's 35x data increase (74k vs 2k rows) with 1 epoch outperforms v3's 2 epochs on 2k rows.
+2. **Cross-domain transfer improved.** m2 shows the highest transfer score in the lineage on its benchmark.
+3. **m2 generates more concisely.** 1,108 chars avg vs base's 1,489 — the model learned to be more focused.
+4. **m2 is faster.** 4.9s avg latency vs base's 6.9s — shorter, more direct responses.
+5. **Cross-benchmark caveat.** v1–v3 were scored by qwen3:8b judge; m2 by heuristic. The two benchmarks are not directly comparable.