library_name: transformers
license: apache-2.0
language:
- eu
- en
- es
tags:
- mamba-2
- basque
- autocomplete
- on-device
- low-resource
pipeline_tag: text-generation
Morpheus v2 (Mamba-2) — Basque Autocomplete
A 91M-parameter Mamba-2 language model for on-device Basque (Euskara) text autocompletion.
Model Details
- Architecture: Mamba-2 (State Space Model)
- Parameters: 91M
- Embedding vocab: 4,000 (Unigram SentencePiece)
- Hidden dimension: 768
- Layers: 24
- State dimension: 64
- Head dimension: 64
- Inner dimension: 1,536
- Sequence length: 1,024
- Training tokens: ~10 billion
- Training steps: 76,000 (best checkpoint at 74,000)
- Held-out PPL: 7.13
- Trained without BOS token
Tokenizer
A 4K Unigram SentencePiece tokenizer trained on the cleaned Basque corpus. The small vocabulary size was chosen based on evidence that lower vocab sizes achieve lower downstream perplexity for agglutinative low-resource languages (cf. QuechuaTok).
add_bos_token: false(the model was trained without a BOS token)- EOS token:
</s>(id=2) - UNK token:
<unk>(id=0)
Intended Use
On-device Basque text autocomplete and predictive keyboard input. The model is small enough to run on CPU via llama.cpp (see the GGUF quantized versions at itzune/morpheus-gguf).
Training Data
Trained on a ~22 GB cleaned Basque text corpus comprising Wikipedia, news (Berria), literature, and other web-crawled sources. The corpus underwent a multi-stage cleaning pipeline (deduplication, language filtering, quality auditing).
Quantized Versions
GGUF quantized models (Q4_K_M, Q5_K_M) for llama.cpp inference are available at: itzune/morpheus-gguf
Citation
@misc{morpheus_v2_mamba,
author = {Xabier Ezpeleta},
title = {Morpheus v2: On-Device Basque Autocompletion with Mamba-2},
year = {2026},
}