| license: apache-2.0 | |
| tags: | |
| - prime-rl | |
| - moe | |
| - test-model | |
| library_name: transformers | |
| <div align="center"> | |
| <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/61e020e4a343274bb132e138/H2mcdPRWtl4iKLd-OYYBc.jpeg" width="200"/> | |
| </div> | |
| # minimax-m2-tiny | |
| A small (~252M parameter) MiniMax M2 MoE model for testing only. It is generally compatible with vLLM and HuggingFace Transformers but is meant to be used with [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl). | |
| This model has random weights (no SFT warmup yet due to a chat template tokenization issue with MiniMax's tokenizer). | |
| ## Quick Start | |
| ```bash | |
| uv run rl @ configs/ci/integration/rl_moe/minimax_m2.toml | |
| ``` | |
| See the [Testing MoE at Small Scale](https://github.com/PrimeIntellect-ai/prime-rl/blob/main/docs/testing-moe-at-small-scale.md) guide for full instructions. | |
| ## Model Details | |
| | Parameter | Value | | |
| |-----------|-------| | |
| | Hidden size | 512 | | |
| | Layers | 12 | | |
| | Experts | 8 | | |
| | Active experts | 4 | | |
| | Parameters | ~252M | | |
| ## Links | |
| - [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) - RL training framework | |
| - [PrimeIntellect](https://www.primeintellect.ai/) - Building infrastructure for decentralized AI | |