Add README

README.md

@@ -0,0 +1,43 @@
+# Asteria Custom Model
+
+Custom MoE model with **Kimi K2.7's trained brain weights**.
+
+## Architecture (matches Kimi K2.7)
+
+- 61 layers (Layer 0 = dense, Layers 1-60 = MoE)
+- 384 experts per MoE layer, 8 active per token
+- 1 shared expert per MoE layer (always active)
+- MLA attention (Multi-head Latent Attention)
+- Hidden size: 7168
+- Vocab: 163,840
+- Total params: ~1 trillion
+- Active per token: ~32 billion
+
+## Features
+
+1. **Sparse loading**: Only 8 experts in RAM per layer (~700 MB)
+2. **Expert training**: Train one expert at a time with LoRA
+3. **Router always trainable**: Whole brain improves when any expert trains
+4. **Dynamic experts**: Add/remove experts on the fly
+
+## Source
+
+Weights transferred from `gamansai/ai/custom_models/Kimi-K2.7-Code/` (originally from `moonshotai/Kimi-K2.7-Code`).
+
+## Usage
+
+```python
+from huggingface_hub import snapshot_download
+import torch, numpy as np
+
+# Download config
+path = snapshot_download(repo_id="gamansai/asteria-custom-model", allow_patterns="config.json")
+
+# Load expert weights (sparse — only fetch what you need)
+path = snapshot_download(
+    repo_id="gamansai/asteria-custom-model",
+    allow_patterns="L01/experts/E000/*"
+)
+# Load gate_proj weights
+raw = open(f"{path}/L01/experts/E000/gate_proj_weight_packed.bin", "rb").read()
+```