---
license: openmdw-1.1
base_model: unsloth/Laguna-S-2.1-GGUF
base_model_relation: quantized
tags:
  - gguf
  - rocmfpx
  - laguna-s-2.1
  - moe
quantized_from: unsloth/Laguna-S-2.1-GGUF (BF16)
---

# Laguna-S-2.1 — ROCmFPX hybrid quant (Q4 experts + Q8 rest)

This is a **quantization of [`unsloth/Laguna-S-2.1-GGUF` (BF16)](https://huggingface.co/unsloth/Laguna-S-2.1-GGUF)**
(the original model is [Poolside's Laguna S 2.1](https://huggingface.co/poolside/Laguna-S-2.1),
a 118B-total / 8B-activated MoE with 256 routed experts).

> [!IMPORTANT]
> **You need the [ROCmFPX fork of llama.cpp](https://github.com/charlie12345/ROCmFPX)** (or a llama.cpp
> build with ROCmFPX support). This file uses the experimental `q4_0_rocmfp4_fast` (type 101) and
> `q8_0_rocmfpx` (type 103) weight formats, which stock llama.cpp releases do **not** understand —
> loading it elsewhere will fail with an unknown tensor type error.

## Recipe

| Tensor group | Type | Count |
|---|---|---:|
| Routed experts: `blk.N.ffn_{gate,up,down}_exps` | `q4_0_rocmfp4_fast` (4.25 bpw) | 141 |
| Everything else quantizable (attention, shared experts, embeddings, output head) | `q8_0_rocmfpx` (8.25 bpw) | 386 |
| Norms, biases, router weights/scales | `f32` (untouched) | 287 |

- **Total size:** ~61.6 GB (4.39 bpw) vs 224 GB BF16 source — single merged file, no shards
- Rationale: routed experts dominate parameters (~97%) and tolerate 4-bit well; everything else
  stays near-lossless at 8-bit.

### How it was made

```bash
# from the ROCmFPX fork (CPU-only build works fine for quantization)
llama-quantize \
  --tensor-type "ffn_(gate|up|down)_exps=q4_0_rocmfp4_fast" \
  Laguna-S-2.1-BF16-00001-of-00005.gguf \
  Laguna-S-2.1-Q8_0_ROCMFPX-Q4FAST-experts.gguf Q8_0_ROCMFPX
# then merged from 5 shards: llama-gguf-split --merge ...
```

Note the leading dense layer (`blk.0`) keeps its dense FFN at `q8_0_rocmfpx` — only the routed
expert tensors were overridden.

## Usage

```bash
# build ROCmFPX for your GPU (see the repo README; e.g. Strix Halo):
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh

# run (Vulkan was fastest in upstream tests on Strix Halo):
./build-strix-rocmfp4/bin/llama-cli \
  -m Laguna-S-2.1-Q8_0_ROCMFPX-Q4FAST-experts.gguf \
  -dev Vulkan0 -ngl 999 -fa on --jinja
```

## Benchmarks

> [!NOTE]
> Benchmarks are pending — placeholder table below.

| Backend / GPU | Prompt (tok/s) | Generation (tok/s) | Context | Notes |
|---|---|---|---|---|
| TBD | TBD | TBD | TBD | TBD |

Quality comparison vs BF16 source (perplexity / KLD): TBD.

## Attribution & license

- Quantized from: [`unsloth/Laguna-S-2.1-GGUF`](https://huggingface.co/unsloth/Laguna-S-2.1-GGUF) (BF16 shards)
- Original model: [Poolside Laguna S 2.1](https://huggingface.co/poolside/Laguna-S-2.1)
- License: `openmdw-1.1` (inherited from the source model)
- Quant formats by the [ROCmFPX project](https://github.com/charlie12345/ROCmFPX)