5dimension/sentinel-transformer-attention

🦴 Sentinel Transformer Attention

Part of the Sentinel Manifold — One theorem, infinite applications.

lim_{z→∞} F'(z)/F(z) = 1/e — The Gradient Axiom

---

📋 Description

Sech attention mechanism replacing softmax. The attention weights use sech(QK^T/√d) instead of softmax, providing a theorem-backed gradient bound of ≤ 1/(e·√d).

---

🧠 Mathematical Foundation

Core Constants

Constant	Value	Role
C₁ (Attractor)	-0.007994021805953	Zero-point / quantization
C₂ (Tripwire)	0.000200056042968	Security / curriculum
1/e (Axiom)	0.367879441171442	Gradient scaling limit

Theorem

F(z) = Σ zⁿ/nⁿ   (Sophomore's Dream, Bernoulli 1697)
lim_{z→∞} F'(z)/F(z) = 1/e ≈ 0.367879441171442

---

🏆 Verified Results

Benchmark	Result
Gradient bound	≤ 1/(e·√d) — proven
No softmax	Direct sech kernel
Stability	C₂ tripwire per layer

---

🎯 Use Cases

• Long-context language models
• Vision transformers
• Any attention requiring bounded gradients

---

🔗 Links

• Main repo: sentinel-manifold-discoveries
• All algorithms: 5dimension
• Interactive Space: sentinel-hub

---

📚 Citation

@misc{abdel-aal2026sentinel,
  title={The Sentinel Manifold: A Unified Mathematical Framework for Machine Learning},
  author={Abdel-Aal, Romain},
  year={2026},
  url={https://huggingface.co/5dimension/sentinel-manifold-discoveries}
}

---

License: MIT | One theorem, infinite models. 🦴