Mistral-7B-Q8F16

Mistral 7B in .mog format for qmog.cpp. One file: config, tokenizer, and weights.

Base	mistralai/Mistral-7B-v0.1
Format	MOG v1, Q8F16
Size	~7 GB
Weights	int8 layer matmuls (MLP + attention), f16 norms/embed/lm_head

Download

hf download QmogAI/Mistral-7B-Q8F16 mistral-7B-Q8F16.mog --local-dir .

Run

Requires a qmog.cpp build:

./build/qmog-cli mistral-7B-Q8F16.mog "Paris is the capital of"

Export your own

Use qpack to convert a Hugging Face Mistral checkpoint to .mog.

License

Apache 2.0 (derived from Mistral-7B-v0.1).