Deep Delta Learning
Paper
•
2601.00417
•
Published
•
29
NN Arch Components
-
-
mHC: Manifold-Constrained Hyper-Connections
Paper • 2512.24880 • Published • 240 -
VersatileFFN: Achieving Parameter Efficiency in LLMs via Adaptive Wide-and-Deep Reuse
Paper • 2512.14531 • Published • 12 -
Stronger Normalization-Free Transformers
Paper • 2512.10938 • Published • 19 -
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Paper • 2505.06708 • Published • 10 -
Transformers without Normalization
Paper • 2503.10622 • Published • 170 -
Forgetting Transformer: Softmax Attention with a Forget Gate
Paper • 2503.02130 • Published • 32 -
Hyper-Connections
Paper • 2409.19606 • Published • 25 -
Virtual Width Networks
Paper • 2511.11238 • Published • 37