SmolLM2 1.7B Instruct (MNN)

Hugging Face's SmolLM2 1.7B Instruct as a 4-bit MNN bundle for chatting on your phone with TokForge.

Published by TokForge for on-device inference.

What it is Chat model
Format MNN bundle, 4-bit weights (block 128)
Runs on Android phones (CPU, OpenCL or Vulkan GPU)
Phone class No catalog class; needs more free RAM than its 1.16 GB of weights
Download 1.17 GB (8 files)
Base model HuggingFaceTB/SmolLM2-1.7B-Instruct
License Apache-2.0

TokForge

Runs on-device in the TokForge app.

Use it in TokForge

  1. Install TokForge from Google Play, or join the iOS TestFlight.
  2. On Android, open Models > Browse, search for SmolLM2-1.7B-Instruct-MNN and download it.
  3. Load it and start a chat. The model runs on the phone; once it is downloaded, chatting needs no connection.

Collections: TokForge Chat Models (MNN).

Files

File Size What it is
config.json 210 B MNN runtime settings
embeddings_bf16.bin 201 MB Token embeddings (bf16)
export_args.json 1.11 KB Export settings used for the conversion
llm.mnn 419 KB Model graph
llm.mnn.json 843 KB Model graph as JSON (reference copy)
llm.mnn.weight 963 MB Model weights
llm_config.json 621 B Model settings and chat template
tokenizer.txt 885 KB Tokenizer

About the model

Hugging Face's own SmolLM2 1.7B Instruct is a compact model designed for on-device use. It was trained on 11 trillion tokens and then instruction-tuned. It suits quick questions and answers, text completion and simple tasks rather than long, deep answers.

Architecture LlamaForCausalLM (24 layers, 2048 hidden)
Parameters 1.7B (4-bit quantized)
Quantization W4A16 (4-bit weights, block size 128)
Vocab 49,152 tokens

Recommended settings

Setting Value
Backend OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback)
Precision Low
Threads 4
Thinking Off

How it was made

Converted using MNN's llmexport pipeline:

python llmexport.py --path HuggingFaceTB/SmolLM2-1.7B-Instruct --export mnn --quant_bit 4 --quant_block 128

Limitations

  • Intended for TokForge / MNN on-device inference on Android
  • This is a runtime bundle, not a standard Transformers training checkpoint
  • Quantization (Q4) may slightly reduce quality compared to the full-precision original

License

SmolLM2 1.7B Instruct is released by Hugging Face under the Apache-2.0 license.

Credits

  • SmolLM2 1.7B Instruct by Hugging Face (HuggingFaceTB) (Hugging Face). Used under the Apache-2.0 license.
  • Converted with llmexport from MNN by Alibaba (Apache-2.0).
Downloads last month
60
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for darkmaniac7/SmolLM2-1.7B-Instruct-MNN

Quantized
(135)
this model

Collection including darkmaniac7/SmolLM2-1.7B-Instruct-MNN