README.md

---
library_name: vllm
inference: false
extra_gated_description: >-
  To learn more about how we process your personal data, please read our <a
  href="https://poolside.ai/legal/privacy">Privacy Policy</a>.
tags:
- laguna-m.1
- vllm
- sglang
- bf16
- moe
license: apache-2.0
pipeline_tag: text-generation
---

<p align="center">
  <img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-m1-banner.svg" width="800px">
</p>

<p align="center">
  <a href="https://poolside.ai/blog/laguna-a-deeper-dive"><strong>Release blog post</strong></a> ·
  <a href="https://poolside.ai/assets/laguna/laguna-m1-xs2-technical-report.pdf"><strong>Technical report</strong></a>
</p>

<br>

# Laguna M.1-base

Laguna M.1-base is the **pre-trained base checkpoint** for [Laguna M.1](https://huggingface.co/poolside/Laguna-M.1), a 225B total parameter Mixture-of-Experts model with 23B activated parameters per token. This is the base model **prior to post-training and reinforcement learning** — it is a text-completion model with no instruction-following, reasoning, or tool-calling behavior. For agentic coding and chat use, use the post-trained [Laguna M.1](https://huggingface.co/poolside/Laguna-M.1).

> [!NOTE]
> For details on how we trained Laguna, check out our [release blog post](https://poolside.ai/blog/laguna-a-deeper-dive) and [technical report](https://poolside.ai/assets/laguna/laguna-m1-xs2-technical-report.pdf).

## Highlights

* **Large sparse MoE**: Laguna M.1 is a 70-layer MoE transformer with 225B total parameters and 23B activated parameters per token
* **High-capacity expert routing**: After 3 dense SwiGLU layers, Laguna M.1 uses 67 sparse MoE layers with 256 experts, top-k=16 routing and auxiliary-loss-free load balancing
* **Global attention architecture**: Laguna M.1 uses global attention across all layers with 64 Q-heads, 8 KV-heads and softplus attention output gating
* **Pre-instruct base checkpoint**: Trained with pre-training only; intended as a starting point for further training rather than direct chat/agentic use
* **Apache 2.0 license**: Use and modify freely for commercial and non-commercial purposes

---

## Model overview

- Stage: pre-training only (no post-training or reinforcement learning)
- Number of parameters: 225B total with 23B activated per token
- Optimizer: Muon
- Layers: 70 layers with global attention
- Experts: 256 experts with 1 shared expert; top-k=16 routing
- Dense layers: first 3 layers are dense SwiGLU; remaining 67 layers are sparse MoE
- Attention: 64 Q-heads, 8 KV-heads, head dimension 128, with softplus attention output gating
- Positional encoding: RoPE with YaRN
- Modality: text-to-text (completion)
- Context window: 262,144 tokens

## Usage

Laguna M.1-base is a text-completion model. It has no chat template, reasoning, or tool-calling support — serve it without the reasoning/tool-call parsers and prompt it with raw text.

### vLLM

Laguna support is available in [vLLM](https://github.com/vllm-project/vllm) (v0.21.0 and later, [vllm-project/vllm#41129](https://github.com/vllm-project/vllm/pull/41129)).

```shell
pip install 'vllm>=0.21.0'

vllm serve \
    --model poolside/Laguna-M.1-base \
    --served-model-name laguna-base
```

Query the completions endpoint from any OpenAI-compatible client:

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

completion = client.completions.create(
    model="laguna-base",
    prompt="def fibonacci(n):\n",
    max_tokens=128,
    temperature=0.7,
)
print(completion.choices[0].text)
```

### SGLang

Laguna M.1-base is supported in SGLang via [sgl-project/sglang#28400](https://github.com/sgl-project/sglang/pull/28400). As a completion model, serve it without the reasoning/tool-call parsers. A full serving recipe will be added here.

### Transformers

Laguna is supported in Transformers `v5.7.0` and later ([huggingface/transformers#45673](https://github.com/huggingface/transformers/pull/45673)).

> [!NOTE]
> Laguna M.1-base is a 225B-parameter model; loading the BF16 checkpoint in Transformers requires substantial multi-GPU memory (`device_map="auto"` shards across available devices). For single-node serving, vLLM is recommended.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "poolside/Laguna-M.1-base"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

inputs = tokenizer("def fibonacci(n):\n", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## License

This model is licensed under the [Apache 2.0 License](https://huggingface.co/poolside/Laguna-M.1-base/blob/main/LICENSE.md).

## Intended and Responsible Use 

Laguna M.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna M.1 is subject to the [Apache 2.0 License](https://huggingface.co/poolside/Laguna-M.1/blob/main/LICENSE.md), and should be used consistently with Poolside's [Acceptable Use Policy](https://poolside.ai/legal/acceptable-use-policy). We advise against circumventing Laguna M.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.

Please report security vulnerabilities or safety concerns to [security@poolside.ai](mailto:security@poolside.ai).