Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,87 @@
+---
+license: apache-2.0
+base_model: PrimeIntellect/INTELLECT-3
+tags:
+  - REAP
+  - pruned
+  - MoE
+  - glm4_moe
+  - Cerebras
+library_name: transformers
+---
+
+# INTELLECT-3-REAP-50
+
+**50% expert-pruned version of [PrimeIntellect/INTELLECT-3](https://huggingface.co/PrimeIntellect/INTELLECT-3) using Cerebras REAP (Router-weighted Expert Activation Pruning).**
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Base Model | PrimeIntellect/INTELLECT-3 (248B MoE) |
+| Architecture | GLM-4 MoE (glm4_moe) |
+| Compression | 50% (64 experts pruned) |
+| Remaining Experts | 64 per layer |
+| Parameters | ~124B |
+| Format | BF16 SafeTensors |
+| Size | 107 GB |
+
+## REAP Configuration
+
+```yaml
+dataset: 0xSero/glm47-reap-calibration-v2
+samples: 1360
+  - evol-codealpaca-v1: 700 (code generation)
+  - xlam-function-calling-60k: 330 (function calling)
+  - SWE-smith-trajectories: 330 (agentic multi-turn)
+distance_measure: angular
+seed: 42
+model_max_length: 2048
+compression_ratio: 0.50
+prune_method: reap
+```
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "0xSero/INTELLECT-3-REAP-50",
+    torch_dtype="auto",
+    device_map="auto",
+    trust_remote_code=True
+)
+tokenizer = AutoTokenizer.from_pretrained("0xSero/INTELLECT-3-REAP-50", trust_remote_code=True)
+
+messages = [{"role": "user", "content": "Write a Python function to calculate fibonacci numbers"}]
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
+outputs = model.generate(inputs, max_new_tokens=512)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Related Models
+
+| Model | Compression | Format | Size |
+|-------|-------------|--------|------|
+| [INTELLECT-3-REAP-50](https://huggingface.co/0xSero/INTELLECT-3-REAP-50) | 50% | BF16 | 107GB |
+| INTELLECT-3-REAP-50-W4A16 | 50% | W4A16 GPTQ | ~30GB (coming soon) |
+
+## Citation
+
+```bibtex
+@article{cerebras2025reap,
+  title={REAP: Router-weighted Expert Activation Pruning for MoE Models},
+  author={Cerebras Systems},
+  year={2025}
+}
+```
+
+## Acknowledgments
+
+- **[Prime Intellect](https://primeintellect.ai/)** - For sponsoring compute and creating INTELLECT-3
+- **[Cerebras](https://www.cerebras.net/)** - For the REAP pruning methodology
+- Pruned using the [Cerebras REAP implementation](https://github.com/Cerebras/reap)
+
+---
+*This model was created as part of efficiency research for large MoE models.*