README.md

---
library_name: gguf
license: apache-2.0
language:
  - eu
  - en
  - es
tags:
  - gguf
  - llama.cpp
  - mamba-2
  - basque
  - autocomplete
  - on-device
  - low-resource
base_model: itzune/morpheus
pipeline_tag: text-generation
---

# Morpheus v2 (Mamba-2) — Basque Autocomplete (GGUF)

GGUF quantized versions of [itzune/morpheus](https://huggingface.co/itzune/morpheus), a 91M-parameter Mamba-2 language model for on-device Basque (Euskara) text autocompletion.

## Available Quantizations

| File | Quant | Size | BPW | Use Case |
|------|-------|------|-----|----------|
| `morpheus-v2-mamba.Q4_K_M.gguf` | Q4_K_M | 55 MB | 4.64 | Recommended for deployment (best size/quality tradeoff) |
| `morpheus-v2-mamba.Q5_K_M.gguf` | Q5_K_M | 66 MB | 5.60 | Higher quality, preserves Basque morphology better |

## Model Details

- **Architecture:** Mamba-2 (State Space Model)
- **Parameters:** 91M
- **Checkpoint:** step 74,000 (best, held-out PPL 7.13)
- **Training tokens:** ~10 billion
- **Tokenizer:** 4K Unigram SentencePiece (included in base model repo)
- **Trained without BOS token**

## Usage with llama.cpp

```bash
# Start the server
llama-server -m morpheus-v2-mamba.Q5_K_M.gguf --host 0.0.0.0 --port 8080 -ngl 0

# Or use the CLI
llama-cli -m morpheus-v2-mamba.Q5_K_M.gguf -p "Kaixo, zer moduz" -n 50
```

### Important: llama.cpp version

Requires llama.cpp built from source **after 2025-07-04** (commit `dc2187d48` or later), which fixes the SSM_SCAN computation for Mamba-2 models with `n_groups > 1`. Earlier builds produce silently incorrect greedy outputs.

## Tokenizer Notes

The SentencePiece tokenizer diverges from llama.cpp's built-in SP tokenizer for this vocabulary. For faithful inference, use token-ID prompts or ensure the demo server's tokenization matches training semantics (no BOS token).

## Inference Engineering

The deployed demo uses several strategies to handle subword tokenization in an interactive keyboard context:
- Retokenization fallback (query shorter prefixes when the typed prefix tokenizes poorly)
- Sticky merge (carry forward candidates from previous keystrokes)
- Top-k exceeding display-k (fetch 5, display 3)
- Next-word candidate extraction

See the accompanying paper for details.

## Citation

```bibtex
@misc{morpheus_v2_mamba,
  author = {Xabier Ezpeleta},
  title = {Morpheus v2: On-Device Basque Autocompletion with Mamba-2},
  year = {2026},
}
```