SUMMARY.md

Half-Life Regularization Experiment Summary

Generated: 2026-01-22T14:32:53.545187

Overview

This experiment suite addresses the half-life collapse problem discovered by Melanie/Tiago:

"After training at GPT-2 scale, oscillator half-lives collapse to ~10 steps."

Key Results

Collapse and Recovery

The half-life regularizer successfully provides gradients to restore long-range oscillators:

• Initial distribution: Log-uniform over [1, 4096]
• Collapsed distribution: All < 10 steps
• After regularization step: Distribution spreads back toward target

Identity Reconstruction

Condition	Verdict	Critical K
Without Regularization	FAIL (GRADUAL DRIFT)	128
With Regularization	PASS (PHASE TRANSITION)	64

Conclusion

Half-life regularization is decisive for long-context coherence.

The experiment confirms:

1. Half-life collapse prevents long-range identity preservation
2. The regularizer restores the capacity for long-context reasoning
3. This validates the hypothesis from Melanie/Tiago's discovery

Files Included

• collapse_recovery.json - Half-life collapse/recovery data
• identity_reconstruction/ - Full experiment results
• PRESENTATION_HALF_LIFE_REGULARIZATION.md - Slides
• all_results.json - Complete results data

Recommendations

1. Integrate HalfLifeRegularizer into FDRA training loss
2. Set lambda1 = 0.01, lambda2 = 0.01 as starting points
3. Monitor half-life histogram during training
4. Test on long-context benchmarks (QA, summarization)

---

Generated by run_half_life_experiment.py