model card: CC-BY-NC-SA 4.0, lineage, dataset curriculum, base/imprint structure

README.md

@@ -0,0 +1,167 @@
+---
+license: cc-by-nc-sa-4.0
+language:
+- en
+tags:
+- ssm
+- state-space-model
+- mamba
+- causal-lm
+- rtaforge
+- anvaya
+---
+
+# Rabbit-RtaSSM — Anvaya 2.7B
+
+**RtaForge Anvaya Series** | Durga fu-64 Architecture | 2.7B Parameters
+
+> Commercial licensing available — contact guha@rtaforge.in
+
+---
+
+## Model Lineage
+
+```
+Mamba2 2.7B
+    │
+    └─▶  Rabbit-RtaSSM 2.7B  (weight subsumination — patent pending)
+             │
+             ├─▶  base/         ← 1,500-step trained base model
+             │    Fine-tuned on: OpenOrca · Cosmopedia · LogiQA · ARC-Challenge ·
+             │                   GSM8K · MetaMathQA · SciQ · Python instructions ·
+             │                   Glaive function-calling · Glaive alignment
+             │
+             └─▶  imprint/      ← base + Rabbit personality SFT
+```
+
+**Weight Subsumination** is a proprietary RtaForge technique for transplanting learned
+representations from a source architecture into a structurally distinct target model.
+*Patent pending — technique details not disclosed.*
+
+---
+
+## Model Description
+
+Rabbit-RtaSSM is a 2.7B parameter State Space Model (SSM) trained by [RtaForge](https://rtaforge.in)
+as part of the **Anvaya** small language model series. It uses the proprietary **Durga fu-64**
+architecture — a custom SSM variant with fortress layers and constitutional governance via the
+Gurukul training framework.
+
+Rabbit is the fast, general-purpose runner of the Anvaya trio (Rabbit · Raccoon · Polar Bear),
+optimised for high-throughput instruction following, logic, math, STEM, and tool dispatch.
+
+### Architecture
+
+| Property | Value |
+|----------|-------|
+| Architecture | Durga fu-64 (custom SSM) |
+| Base lineage | Mamba2 2.7B (weight subsumination) |
+| Parameters | ~2.7B |
+| Tokenizer | EleutherAI/gpt-neox-20b (vocab 50,280) |
+| Sequence length | 512 |
+| Optimizer | Lion (lr 1e-5) |
+| Training framework | Gurukul Phase 2 Hardened |
+
+---
+
+## Training Curriculum
+
+Two campaigns on an NVIDIA L4 GPU (Ace Cloud):
+
+### Campaign 1 — 8 phases, ~15,000 steps
+
+| Phase | Steps | Dataset | Focus |
+|-------|-------|---------|-------|
+| 0 | 1,500 | OpenOrca + Cosmopedia | General warmup |
+| 1 | 3,000 | LogiQA + ARC-Challenge | Logic & reasoning |
+| 2 | 2,500 | GSM8K + MetaMathQA | Mathematics |
+| 3 | 2,000 | SciQ | Science / STEM |
+| 4 | 1,500 | Python instructions | Coding |
+| 5 | 1,000 | Glaive function-calling | Tool use |
+| 6 | 2,000 | Glaive alignment | Alignment |
+| 7 | 1,500 | Glaive alignment | Alignment |
+
+### Campaign 2 — Scholar Sprint, 1,500 steps
+
+Phase 5 saturation (Logic Giants corpus), Lion lr=1e-5.
+Final base checkpoint: **Step 1,500**.
+
+---
+
+## Evaluation Results
+
+Evaluated using scale-invariant metrics (Top-K accuracy, Mean Reciprocal Rank)
+vs. random-initialised baseline. 100 samples per corpus, seq_len=512.
+
+| Corpus | Metric | Random Init | Trained | Gain |
+|--------|--------|-------------|---------|------|
+| Biology | Top-1 Accuracy | baseline | **10× baseline** | +10× |
+| Chemistry | Top-1 Accuracy | baseline | **10× baseline** | +10× |
+| Deep Math | MRR | 0.008 | **0.186** | **+22×** |
+
+*Full Step 1,500 evaluation results will be added upon final publication.*
+
+---
+
+## Repository Structure
+
+```
+RtaForge/Anvaya-Raccoon2.7B
+├── base/
+│   └── pytorch_model.bin     ← base model weights (step 1,500)
+├── imprint/
+│   └── pytorch_model.bin     ← base + Rabbit personality SFT
+└── logs/
+    └── training_logs_1500.zip
+```
+
+---
+
+## Usage
+
+This model uses a custom SSM architecture and requires the RtaForge inference stack.
+Standard HuggingFace `AutoModel` is not supported.
+
+```python
+# Requires: rtaforge-substrates + torch, transformers
+from white_rabbit.rabbit_model import create_rabbit_model
+from transformers import AutoTokenizer
+import torch
+
+model = create_rabbit_model(vocab_size=50280, durga_variant="fu-64")
+sd = torch.load("base/pytorch_model.bin", map_location="cpu")
+model.load_state_dict(sd, strict=False)
+model.eval()
+
+tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
+```
+
+---
+
+## License
+
+The model weights in this repository are licensed under
+**Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)**.
+
+- ✅ Free for research, education, and non-commercial use
+- ✅ Derivatives must carry the same licence
+- ❌ Commercial use requires a separate agreement
+
+> **Commercial licensing available — contact guha@rtaforge.in**
+
+---
+
+## Citation
+
+```
+@misc{rtaforge2026rabbit,
+  title  = {Rabbit-RtaSSM: Anvaya 2.7B State Space Model},
+  author = {RtaForge},
+  year   = {2026},
+  url    = {https://huggingface.co/RtaForge/Anvaya-Raccoon2.7B}
+}
+```
+
+---
+
+*Forged at RtaForge — ऋत्*