Morpheus v2 (Mamba-2) — Basque Autocomplete

A 91M-parameter Mamba-2 language model for on-device Basque (Euskara) text autocompletion.

Model Details

• Architecture: Mamba-2 (State Space Model)
• Parameters: 91M
• Embedding vocab: 4,000 (Unigram SentencePiece)
• Hidden dimension: 768
• Layers: 24
• State dimension: 64
• Head dimension: 64
• Inner dimension: 1,536
• Sequence length: 1,024
• Training tokens: ~10 billion
• Training steps: 76,000 (best checkpoint at 74,000)
• Held-out PPL: 7.13
• Trained without BOS token

Tokenizer

A 4K Unigram SentencePiece tokenizer trained on the cleaned Basque corpus. The small vocabulary size was chosen based on evidence that lower vocab sizes achieve lower downstream perplexity for agglutinative low-resource languages (cf. QuechuaTok).

• add_bos_token: false (the model was trained without a BOS token)
• EOS token: </s> (id=2)
• UNK token: <unk> (id=0)

Intended Use

On-device Basque text autocomplete and predictive keyboard input. The model is small enough to run on CPU via llama.cpp (see the GGUF quantized versions at itzune/morpheus-gguf).

Training Data

Trained on a ~22 GB cleaned Basque text corpus comprising Wikipedia, news (Berria), literature, and other web-crawled sources. The corpus underwent a multi-stage cleaning pipeline (deduplication, language filtering, quality auditing).

Quantized Versions

GGUF quantized models (Q4_K_M, Q5_K_M) for llama.cpp inference are available at: itzune/morpheus-gguf

Citation

@misc{morpheus_v2_mamba,
  author = {Xabier Ezpeleta},
  title = {Morpheus v2: On-Device Basque Autocompletion with Mamba-2},
  year = {2026},
}