Upload Chimera 510M at step 229000

README.md

@@ -8,8 +8,11 @@ tags:
   - hybrid-architecture
 language:
   - en
+thumbnail: auron_banner.png
 ---
 
+![Auron](auron_banner.png)
+
 # Auron-510M
 
 **Auron** — Chimera hybrid GDN-Attention language models with Ouroboros weight sharing.
@@ -28,12 +31,8 @@ language:
 - **GDN:Attn ratio:** 3:1 (every 4th layer is attention)
 - **Virtual equivalent:** ~1,020,434,560 params
 
-## Training Curves
-
-![Training Curves](training_curves.png)
-
 ## Training
-- **Step:** 225,500
+- **Step:** 229,000
 - **Data:** Mixed (75% FineWeb-Edu, 18% StarCoder, 5% FineMath, 2% UltraChat)
 - **Optimizer:** Muon + AdamW (decoupled embedding LR)
 - **Schedule:** WSD (Warmup-Stable-Decay)
@@ -41,7 +40,7 @@ language:
 ## Usage
 
 ```bash
-pip install ouro
+git clone https://github.com/Fy-/Auron && cd Auron && rye sync
 ```
 
 ```python
@@ -53,6 +52,12 @@ generate(model, tokenizer, device, "The history of")
 
 ## Sampling
 
-Default: T=0.7, top_k=20, top_p=0.9, rep_pen=1.0, presence_pen=1.5 (Ouroboros requires presence penalty >= 1.5 to prevent attractor wells).
+Default: T=0.7, top_k=20, top_p=0.9, rep_pen=1.0, presence_pen=1.5 (Ouroboros weight sharing requires presence penalty >= 1.5 to prevent attractor wells).
+
+## Links
+
+- **Paper:** [Auron: Depth-Efficient Language Models via Hybrid Recurrent-Attention Weight Sharing](https://github.com/Fy-/Auron/blob/master/Auron_chimera_topology_paper.pdf)
+- **Code:** [github.com/Fy-/Auron](https://github.com/Fy-/Auron)
+- **Models:** [huggingface.co/nyxia](https://huggingface.co/nyxia)
 
 Built by [Florian Gasquez](https://fyx.jp) ([@nyxia](https://huggingface.co/nyxia)). Part of the [Soulkyn](https://soulkyn.com) project.

config.json

@@ -26,7 +26,7 @@
   "architecture": "Chimera",
   "config_class": "ChimeraConfig",
   "topology": "4 bottom + 4x3 top = 16 virtual",
-  "step": 225500,
+  "step": 229000,
   "total_params": 510217280,
   "size_label": "510M",
   "model_type": "zara-ml"

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9bac19b2677ba1d1d5cf9f44b1f711f15ba0110a25227419d61ac213ffbda735
+oid sha256:ed1d709dc0fbbe9152d1eb828d8c6010e380d0fdfd62dbf28bbf1332e97348b0
 size 1487197656