reports/COMPARE_JACKRONG.md

# Kuiper-R1 vs Jackrong/Trace-Inverter-4B — trace-inversion head-to-head

Same held-out benchmark (`DeepSeek-V4-Distill-8000x`, inverted), same scorer, n=100. Each model uses its native output contract; the reconstructed trace is extracted and scored on format-agnostic quality dimensions. **Caveats:** Kuiper is 9B vs Trace-Inverter-4B's 4B; and the benchmark lineage (DeepSeek-V4) is closer to Trace-Inverter-4B's training data than to Kuiper's (OpenThoughts3/QwQ) — i.e. roughly Jackrong's home turf.

| Metric | kuiper (kuiper-r1-orpo) | jackrong (jackrong) |
|---|---|---|
| expansion quality | 0.8149 | 0.7865 |
| semantic consistency w/ bubbles | 0.5601 | 0.4732 |
| trace vs reference similarity | 0.1818 | 0.1411 |
| depth-to-concision | 0.3403 | 0.2671 |
| no invented tools | 1.0 | 0.98 |
| no repetition collapse | 0.98 | 0.84 |
| non-empty trace | 1.0 | 1.0 |
| mean trace chars | 5364.64 | 3636.09 |
## Final verdict (Kuiper rp=1.1 vs Jackrong-4B)

With repetition_penalty=1.1 at inference, **Kuiper-R1 beats Trace-Inverter-4B on every dimension
that matters** (depth-to-concision excluded per objective — we favor CoT quality over brevity):
- no repetition collapse: **0.98 vs 0.84**
- expansion quality: **0.815 vs 0.786**
- semantic consistency w/ bubbles: **0.560 vs 0.473**
- reference-trace similarity: **0.182 vs 0.141**
- no invented tools: **1.00 vs 0.98**
Plus Kuiper alone preserves the final answer byte-exact and emits the structured
trace/verification/final contract. Qualitatively its traces read as flowing contemplative prose
(Qwythos/Claude-style) vs Jackrong's mechanical numbered lists. Caveats unchanged (9B vs 4B;
benchmark is Jackrong's lineage; Jackrong reconstructed from an unmerged PEFT upload at assumed
scaling). Worked examples: reports/EXAMPLES.md.