--- library_name: transformers inference: false extra_gated_description: >- To learn more about how we process your personal data, please read our Privacy Policy. tags: - laguna-m.1 - moe license: apache-2.0 pipeline_tag: text-generation ---

poolside-banner

Release blog post ยท Technical report


# Laguna M.1-base Laguna M.1-base is the **pre-trained base checkpoint** for [Laguna M.1](https://huggingface.co/poolside/Laguna-M.1), a 225B total parameter Mixture-of-Experts model with 23B activated parameters per token. This is the base model prior to post-training and reinforcement learning; for agentic coding and instruction following, use the post-trained [Laguna M.1](https://huggingface.co/poolside/Laguna-M.1). ## Model overview - Number of parameters: 225B total with 23B activated per token - Optimizer: Muon - Layers: 70 layers with global attention - Experts: 256 experts with 1 shared expert; top-k=16 routing - Dense layers: first 3 layers are dense SwiGLU; remaining 67 layers are sparse MoE - Attention: 64 Q-heads, 8 KV-heads, head dimension 128, with softplus attention output gating - Positional encoding: RoPE with YaRN - Modality: text-to-text - Context window: 262,144 tokens ## License This model is licensed under the [Apache 2.0 License](https://huggingface.co/poolside/Laguna-M.1-base/blob/main/LICENSE.md). ## Intended and Responsible Use Laguna M.1-base is a base (pre-instruct) model intended for research and as a starting point for further training; you are responsible for confirming that it is appropriate for your intended application. It is subject to the [Apache 2.0 License](https://huggingface.co/poolside/Laguna-M.1-base/blob/main/LICENSE.md), and should be used consistently with Poolside's [Acceptable Use Policy](https://poolside.ai/legal/acceptable-use-policy). Please report security vulnerabilities or safety concerns to [security@poolside.ai](mailto:security@poolside.ai).