library_name: transformers
inference: false
extra_gated_description: >-
To learn more about how we process your personal data, please read our <a
href="https://poolside.ai/legal/privacy">Privacy Policy</a>.
tags:
- laguna-m.1
- moe
license: apache-2.0
pipeline_tag: text-generation
Release blog post · Technical report
Laguna M.1-base
Laguna M.1-base is the pre-trained base checkpoint for Laguna M.1, a 225B total parameter Mixture-of-Experts model with 23B activated parameters per token. This is the base model prior to post-training and reinforcement learning; for agentic coding and instruction following, use the post-trained Laguna M.1.
Model overview
- Number of parameters: 225B total with 23B activated per token
- Optimizer: Muon
- Layers: 70 layers with global attention
- Experts: 256 experts with 1 shared expert; top-k=16 routing
- Dense layers: first 3 layers are dense SwiGLU; remaining 67 layers are sparse MoE
- Attention: 64 Q-heads, 8 KV-heads, head dimension 128, with softplus attention output gating
- Positional encoding: RoPE with YaRN
- Modality: text-to-text
- Context window: 262,144 tokens
License
This model is licensed under the Apache 2.0 License.
Intended and Responsible Use
Laguna M.1-base is a base (pre-instruct) model intended for research and as a starting point for further training; you are responsible for confirming that it is appropriate for your intended application. It is subject to the Apache 2.0 License, and should be used consistently with Poolside's Acceptable Use Policy.
Please report security vulnerabilities or safety concerns to security@poolside.ai.