itzune/morpheus-gguf

Morpheus v2 (Mamba-2) — Basque Autocomplete (GGUF)

GGUF quantized versions of itzune/morpheus, a 91M-parameter Mamba-2 language model for on-device Basque (Euskara) text autocompletion.

Available Quantizations

File	Quant	Size	BPW	Use Case
step_0074000.Q4_K_M.gguf	Q4_K_M	55 MB	4.64	Recommended for deployment (best size/quality tradeoff)
step_0074000.Q5_K_M.gguf	Q5_K_M	66 MB	5.60	Higher quality, preserves Basque morphology better

Model Details

• Architecture: Mamba-2 (State Space Model)
• Parameters: 91M
• Checkpoint: step 74,000 (best, held-out PPL 7.13)
• Training tokens: ~10 billion
• Tokenizer: 4K Unigram SentencePiece (included in base model repo)
• Trained without BOS token

Usage with llama.cpp

# Start the server
llama-server -m step_0074000.Q5_K_M.gguf --host 0.0.0.0 --port 8080 -ngl 0

# Or use the CLI
llama-cli -m step_0074000.Q5_K_M.gguf -p "Kaixo, zer moduz" -n 50

Important: llama.cpp version

Requires llama.cpp built from source after 2025-07-04 (commit dc2187d48 or later), which fixes the SSM_SCAN computation for Mamba-2 models with n_groups > 1. Earlier builds produce silently incorrect greedy outputs.

Tokenizer Notes

The SentencePiece tokenizer diverges from llama.cpp's built-in SP tokenizer for this vocabulary. For faithful inference, use token-ID prompts or ensure the demo server's tokenization matches training semantics (no BOS token).

Inference Engineering

The deployed demo uses several strategies to handle subword tokenization in an interactive keyboard context:

• Retokenization fallback (query shorter prefixes when the typed prefix tokenizes poorly)
• Sticky merge (carry forward candidates from previous keystrokes)
• Top-k exceeding display-k (fetch 5, display 3)
• Next-word candidate extraction

See the accompanying paper for details.

Citation

@misc{morpheus_v2_mamba,
  author = {Xabier Ezpeleta},
  title = {Morpheus v2: On-Device Basque Autocompletion with Mamba-2},
  year = {2026},
}