DeepSeek R1 Distill Llama 8B (MNN)
DeepSeek's R1 Distill Llama 8B reasoning model as a 4-bit MNN bundle for chatting on your phone with TokForge.
Published by TokForge for on-device inference.
| What it is | Reasoning chat model |
| Format | MNN bundle, 4-bit weights (block 128) |
| Runs on | Android phones (CPU, OpenCL or Vulkan GPU) |
| Phone class | No catalog class; needs more free RAM than its 5.27 GB of weights |
| Download | 5.28 GB (10 files) |
| Base model | deepseek-ai/DeepSeek-R1-Distill-Llama-8B |
| License | MIT (DeepSeek); Llama base under the Llama 3.1 Community License |
TokForge
- Website: https://tokforge.ai
- Discord: https://discord.gg/Acv3CBtfVm
- Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
- iOS TestFlight: https://testflight.apple.com/join/jnufjzRr
Runs on-device in the TokForge app.
Use it in TokForge
- Install TokForge from Google Play, or join the iOS TestFlight.
- On Android, open Models > Browse, search for
DeepSeek-R1-Distill-Llama-8B-MNNand download it. - Load it and start a chat. The model runs on the phone; once it is downloaded, chatting needs no connection.
Collections: TokForge Chat Models (MNN), TokForge Llama, Mistral and Dolphin (MNN).
Files
| File | Size | What it is |
|---|---|---|
config.json |
210 B | MNN runtime settings |
embeddings_bf16.bin |
1.05 GB | Token embeddings (bf16) |
export_args.json |
1.14 KB | Export settings used for the conversion |
llm.mnn |
558 KB | Model graph |
llm.mnn.json |
1.12 MB | Model graph as JSON (reference copy) |
llm.mnn.weight |
4.22 GB | Model weights |
llm_config.json |
2.54 KB | Model settings and chat template |
tokenizer.txt |
1.41 MB | Tokenizer |
LICENSE |
7.63 KB | Llama 3.1 Community License (Meta's text, for the Llama base) |
NOTICE |
116 B | Llama 3.1 notice (Meta) |
About the model
DeepSeek's R1 reasoning capability distilled into a Llama 3.1 8B body: DeepSeek fine-tuned Llama 3.1 8B on reasoning samples curated with DeepSeek-R1. It writes out its thinking step by step before the answer, which suits math, logic puzzles, coding and multi-step analysis.
| Architecture | Llama 3.1 (standard attention, 32 layers, GQA, distilled from DeepSeek-R1) |
| Parameters | 8B (4-bit quantized) |
| Quantization | W4A16 (4-bit weights, block size 128) |
| Vocab | 128,256 tokens |
Recommended settings
| Setting | Value |
|---|---|
| Backend | OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback) |
| Precision | Low |
| Threads | 4 |
| Thinking | On (the chat template always opens a <think> block) |
DeepSeek recommends a temperature between 0.5 and 0.7 (0.6 recommended) and no system prompt: put all instructions in your message.
How it was made
Converted using MNN's llmexport pipeline:
python llmexport.py --path deepseek-ai/DeepSeek-R1-Distill-Llama-8B --export mnn --quant_bit 4 --quant_block 128
Limitations
- Intended for TokForge / MNN on-device inference on Android
- This is a runtime bundle, not a standard Transformers training checkpoint
- Quantization (Q4) may slightly reduce quality compared to the full-precision original
- The reasoning comes before every answer, so replies are longer than from a plain chat model
License
DeepSeek-R1-Distill-Llama-8B is released by DeepSeek under the MIT License.
It is derived from Meta's Llama 3.1 8B, which is licensed under the
Llama 3.1 Community License.
Meta's license text is in LICENSE and the required notice is in NOTICE. Built with Llama.
Credits
- DeepSeek-R1-Distill-Llama-8B by DeepSeek (Hugging Face). Used under the MIT license.
- Llama 3.1 8B by Meta (Hugging Face). Used under the Llama 3.1 Community License. The distilled weights derive from it.
- Built with Llama. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. The license text and this notice are in the
LICENSEandNOTICEfiles in this repo. - Converted with llmexport from MNN by Alibaba (Apache-2.0).
- Downloads last month
- 70
Model tree for darkmaniac7/DeepSeek-R1-Distill-Llama-8B-MNN
Base model
deepseek-ai/DeepSeek-R1-Distill-Llama-8B