| --- |
| library_name: transformers |
| inference: false |
| extra_gated_description: >- |
| To learn more about how we process your personal data, please read our <a |
| href="https://poolside.ai/legal/privacy">Privacy Policy</a>. |
| tags: |
| - laguna-m.1 |
| - moe |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| --- |
| |
| <p align="center"> |
| <img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-m1-banner.svg" width="800px"> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://poolside.ai/blog/laguna-a-deeper-dive"><strong>Release blog post</strong></a> · |
| <a href="https://poolside.ai/assets/laguna/laguna-m1-xs2-technical-report.pdf"><strong>Technical report</strong></a> |
| </p> |
|
|
| <br> |
|
|
| # Laguna M.1-base |
|
|
| Laguna M.1-base is the **pre-trained base checkpoint** for [Laguna M.1](https://huggingface.co/poolside/Laguna-M.1), a 225B total parameter Mixture-of-Experts model with 23B activated parameters per token. This is the base model prior to post-training and reinforcement learning; for agentic coding and instruction following, use the post-trained [Laguna M.1](https://huggingface.co/poolside/Laguna-M.1). |
|
|
| ## Model overview |
|
|
| - Number of parameters: 225B total with 23B activated per token |
| - Optimizer: Muon |
| - Layers: 70 layers with global attention |
| - Experts: 256 experts with 1 shared expert; top-k=16 routing |
| - Dense layers: first 3 layers are dense SwiGLU; remaining 67 layers are sparse MoE |
| - Attention: 64 Q-heads, 8 KV-heads, head dimension 128, with softplus attention output gating |
| - Positional encoding: RoPE with YaRN |
| - Modality: text-to-text |
| - Context window: 262,144 tokens |
|
|
| ## License |
|
|
| This model is licensed under the [Apache 2.0 License](https://huggingface.co/poolside/Laguna-M.1-base/blob/main/LICENSE.md). |
|
|
| ## Intended and Responsible Use |
|
|
| Laguna M.1-base is a base (pre-instruct) model intended for research and as a starting point for further training; you are responsible for confirming that it is appropriate for your intended application. It is subject to the [Apache 2.0 License](https://huggingface.co/poolside/Laguna-M.1-base/blob/main/LICENSE.md), and should be used consistently with Poolside's [Acceptable Use Policy](https://poolside.ai/legal/acceptable-use-policy). |
|
|
| Please report security vulnerabilities or safety concerns to [security@poolside.ai](mailto:security@poolside.ai). |
|
|
