Metis-1.3 Base

Metis-1.3 Base is a 201M-parameter English-first decoder model from the Metis family. It uses a hybrid Mamba2 plus attention backbone and was trained as the base model for later chat and reasoning post-training.

Model summary

Family: Metis
Stage: Base pretraining checkpoint exported for Hugging Face
Parameters: 201,490,560
Architecture: Mamba2-attention hybrid decoder
Context length: 4096
Vocabulary size: 8192
Dtype: bfloat16 weights

Architecture

Metis-1.3 uses 28 decoder blocks:

21 Mamba2 blocks
7 attention blocks at layers 3, 7, 11, 15, 19, 23, and 27
Grouped-query attention with 18 query heads and 6 KV heads
Tied input and output embeddings
RMSNorm

This release is intended as the raw pretrained base for further instruction tuning and reasoning tuning.

Training

Pretraining target: 12B train tokens
Tokenizer: custom 8k tokenizer trained on an 8M-document sample
Intended data mix: English-first web, educational, math, and code-heavy mixture

Intended use

This checkpoint is mainly useful for:

research and experimentation on small hybrid Mamba models
base-model comparison against the chat and think variants
continued fine-tuning or alignment work

It is not the most user-friendly conversational variant. For direct assistant use, the chat or think releases are likely better starting points.

Limitations

This is a small model and will still make factual, reasoning, and instruction-following mistakes.
The eval setup used in this project is lightweight and should be treated as a smoke test rather than a comprehensive benchmark.
English is the intended primary language.

Files

model.safetensors
config.json
generation_config.json
tokenizer.json
tokenizer_config.json
special_tokens_map.json

License

This release inherits the licensing and attribution obligations of the upstream training data sources used in the Metis pipeline. Review dataset licenses and usage constraints before production use.