fractal-agi/fdra-half-life-regularization

FDRA Half-Life Regularization V3: Bug-Fixed Implementation

Date: 2026-01-22
Repository: fractal-agi/fdra-half-life-regularization

Summary

This package contains the bug-fixed implementation of the half-life regularizer for FDRA oscillators, addressing critical issues discovered during code review.

The Problem

FDRA models trained at GPT-2 scale experience half-life collapse: all oscillator decay parameters converge to short values (τ ≈ 2-10 steps), losing the ability to maintain long-range context.

The Solution

A half-life regularizer that maintains a log-uniform distribution of τ ∈ [1, L] where L is the sequence length, ensuring some oscillators can attend to the full context.

Bug Fixes (V3)

Bug	Severity	Issue	Fix
np.clip argument order	CRITICAL	np.clip(x, max, min) clips everything to min	Swapped to (min, max)
Missing tau bounds	CRITICAL	Moment-matching created pathological τ<1	Added compute_bounds_loss()
Sigmoid overflow	Medium	exp(-k*tau) overflowed	Added np.clip(x, -500, 500)
Learning rate	Medium	lr=0.3 overshot valid λ range	Changed to lr=0.0001
Mean-only convergence	Medium	All τ converged to same value	Use log-uniform init directly

Results

Before Bug Fixes (V1-V2)

Regularized tau: [0.48, 6931.1]  ← PATHOLOGICAL
23/32 oscillators with τ < 1     ← WORSE than collapsed!
Basin width: 256 tokens

After Bug Fixes (V3)

Regularized tau: [1.0, 4096.0]   ← Proper log-uniform spread
3/32 oscillators with τ > 2048   ← Long-range coverage
Basin width: 1024 tokens         ← 4x improvement

Identity Reconstruction Comparison

Condition	Verdict	Basin Width	Notes
Collapsed (no regularization)	FAIL	0	Identity immediately lost
Regularized (log-uniform τ)	PARTIAL	1024 (25% of L)	Identity preserved to K=1024

Files

├── half_life_regularizer.py     # Core regularizer with bounds constraint
├── fdra_oscillators.py          # Oscillator bank implementation
├── identity_reconstruction_experiment_v2.py  # Fixed diagnostic experiment
├── identity_v2_*.json           # Raw experimental results
├── IDENTITY_V2_REPORT_*.md      # Generated reports
├── BUGFIX_REPORT.md             # Detailed bug analysis
└── IMPLICATIONS.md              # Research implications

Usage

from half_life_regularizer import HalfLifeRegularizer, HalfLifeRegularizerConfig

config = HalfLifeRegularizerConfig(
    sequence_length=4096,
    tau_min=1.0,
    tau_max=4096.0,
    lambda1=0.01,  # Log-uniform prior weight
    lambda2=0.01,  # Long-tail weight
    lambda3=0.1    # Bounds constraint weight (NEW!)
)

regularizer = HalfLifeRegularizer(config)

# During training:
loss, metrics = regularizer.compute(oscillator_lambdas)
total_loss = task_loss + loss

Citation

If you use this work, please cite:

@misc{fdra-half-life-regularization-2026,
  title={Half-Life Regularization for FDRA Oscillators: Preventing Decay Collapse},
  author={Fractal AGI},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/fractal-agi/fdra-half-life-regularization}
}

License

Apache 2.0