FLAN-T5 base (MNN)
Google's FLAN-T5 base as a 4-bit MNN encoder-decoder bundle that TokForge uses as its on-device summarizer for answers about your documents.
Published by TokForge for on-device inference.
| What it is | Summarizer (encoder-decoder helper model) |
| Format | MNN encoder-decoder bundle (3 graphs), 4-bit weights (block 128) |
| Runs on | Android phones (CPU); also listed in TokForge for iOS |
| Phone class | 4 GB RAM or more (TokForge helper module) |
| Download | 487 MB (10 files) |
| Base model | google/flan-t5-base |
| License | Apache-2.0 |
TokForge
- Website: https://tokforge.ai
- Discord: https://discord.gg/Acv3CBtfVm
- Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
- iOS TestFlight: https://testflight.apple.com/join/jnufjzRr
Runs on-device in the TokForge app.
Use it in TokForge
- Install TokForge from Google Play, or join the iOS TestFlight.
- On Android, open Settings > Speed & engine > Models & Modules and download flan-t5-base Summarizer. TokForge can also fetch it by itself with its other helper files: when you attach a document, or once you allow automatic helper downloads.
- Once it is installed, TokForge sets it as the default summarizer. When TokForge answers a question about your attached documents by combining several passages, it can write that answer with this model, so the chat model is not tied up. It runs on the phone.
- On iOS, it is listed as FLAN-T5 Base Summarizer (MNN).
Collections: TokForge Voice and Helper Models.
Files
| File | Size | What it is |
|---|---|---|
config.json |
1.40 KB | T5 model config from google/flan-t5-base |
decoder.mnn |
173 MB | Decoder for the first output token (4-bit) |
decoder_with_past.mnn |
166 MB | Decoder for later tokens, with cache (4-bit) |
encoder.mnn |
145 MB | Encoder graph and weights (4-bit) |
generation_config.json |
147 B | Generation defaults |
llm_config.json |
934 B | TokForge seq2seq settings (files, quantization, input prefix) |
special_tokens_map.json |
2.20 KB | Special tokens |
spiece.model |
792 KB | SentencePiece tokenizer model |
tokenizer.json |
2.42 MB | Tokenizer |
tokenizer_config.json |
2.54 KB | Tokenizer settings |
The model is split the usual way for encoder-decoder inference: the encoder reads the input once, decoder.mnn writes
the first output token and decoder_with_past.mnn writes the rest, reusing its cache.
About the model
FLAN-T5 base is Google's instruction-tuned T5 encoder-decoder model (about 250M parameters). TokForge gives it text with
the summarize: prefix, as set in llm_config.json.
| Architecture | T5 encoder-decoder (12 encoder and 12 decoder layers, 768 hidden, 12 heads) |
| Quantization | 4-bit weights, block size 128, asymmetric |
| Vocab | 32,128 tokens (SentencePiece) |
| Input length | Up to 512 tokens |
How it was made
Exported as three ONNX graphs (encoder, decoder, and decoder with past), then each converted with MNN's converter to 4-bit
asymmetric weights with block size 128 (llm_config.json records q4-block128-asymmetric).
The tokenizer files (spiece.model, tokenizer.json, tokenizer_config.json, special_tokens_map.json) and
config.json / generation_config.json come from google/flan-t5-base.
Limitations
- Built for TokForge's summarizer route; it is not a chat model
- Inputs are limited to 512 tokens
- Quantization (Q4) may slightly reduce quality compared to the full-precision original
License
FLAN-T5 base is released by Google under the Apache-2.0 license.
Credits
- FLAN-T5 base by Google (Hugging Face). Used under the Apache-2.0 license.
- Converted to MNN with MNN by Alibaba (Apache-2.0).
- Downloads last month
- 995
Model tree for darkmaniac7/flan-t5-base-MNN
Base model
google/flan-t5-base