TokForge Acceleration Pack Qwen3.5 0.8B draft (MNN)

A deprecated Qwen3.5 0.8B draft model for speculative decoding, kept on the Hub for reproducibility; TokForge no longer uses it.

Published by TokForge for on-device inference.

What it is Draft model for speculative decoding (deprecated)
Format MNN bundle, 4-bit weights (block 64)
Runs on Android phones (CPU), MNN runtime; not used by the current TokForge app
Phone class No catalog class; needs more free RAM than its 470 MB of weights
Download 484 MB (7 files)
Base model Qwen/Qwen3.5-0.8B
License Apache 2.0

TokForge

Runs on-device in the TokForge app.

Use it in TokForge

  1. Install TokForge from Google Play.
  2. TokForge does not offer or download this repo. For Qwen3 chat models, open Settings > Speed & engine > Models & Modules and download Acceleration Pack: Qwen3-0.6B 40k (TokForge-AccelerationPack-Draft) instead.
  3. Qwen3.5 chat models need no separate draft: TokForge's Qwen3.5 MNN downloads carry an optional MTP head (the model's own multi-token prediction layer), for example darkmaniac7/Qwen3.5-4B-MNN.

This repo is for TokForge / MNN users who specifically want to reproduce the older Qwen3.5-0.8B draft path.

Collections: TokForge Draft Models.

Files

File Size What it is
config.json 342 B MNN runtime settings
export_args.json 1.10 KB Export settings used for the conversion
llm.mnn 2.15 MB Model graph
llm.mnn.json 5.34 MB Model graph as JSON (reference copy)
llm.mnn.weight 470 MB Model weights
llm_config.json 8.69 KB Model settings and chat template
tokenizer.txt 6.47 MB Tokenizer

Why it is deprecated

Deprecated: this Qwen3.5-0.8B draft bundle is preserved for reproducibility, but the newer Qwen3-0.6B TokForge draft line is the practical default.

  • Cross-family draft pairing does not attach: Qwen3.5 and Qwen3 use different vocabularies (248,320 vs 151,936), so this 0.8B draft cannot pair with Qwen3 targets, and the Qwen3-0.6B drafts cannot pair with Qwen3.5 targets.
  • Qwen3.5 models use linear attention. TokForge keeps speculative decoding with a separate draft model off for them and uses their MTP head instead.

Use instead

How it was made

Exported with MNN's llmexport from huihui-ai's abliterated Qwen3.5 0.8B (huihui-ai/Huihui-Qwen3.5-0.8B-abliterated) as a text-only model (the vision part switched off, is_visual=false), with 4-bit weights in blocks of 64 and tied embeddings. export_args.json has the full export settings.

Limitations

  • Deprecated for normal use.
  • Cross-family drafting is not possible (vocabulary mismatch), and TokForge does not use separate draft models with Qwen3.5 targets.
  • Keep using this only if you need reproducibility for old experiments.
  • This is an MNN runtime bundle, not a standard Transformers checkpoint.

License

Apache License 2.0 (text), the license of Qwen3.5 0.8B.

Credits

  • Qwen3.5-0.8B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
  • Huihui-Qwen3.5-0.8B-abliterated by huihui-ai (Hugging Face). Used under the Apache-2.0 license.
  • Converted with llmexport from MNN by Alibaba (Apache-2.0).
Downloads last month
73
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for darkmaniac7/TokForge-AccelerationPack-Qwen35-Draft

Finetuned
(989)
this model

Collection including darkmaniac7/TokForge-AccelerationPack-Qwen35-Draft