FLAN-T5 base (MNN)

Google's FLAN-T5 base as a 4-bit MNN encoder-decoder bundle that TokForge uses as its on-device summarizer for answers about your documents.

Published by TokForge for on-device inference.

What it is Summarizer (encoder-decoder helper model)
Format MNN encoder-decoder bundle (3 graphs), 4-bit weights (block 128)
Runs on Android phones (CPU); also listed in TokForge for iOS
Phone class 4 GB RAM or more (TokForge helper module)
Download 487 MB (10 files)
Base model google/flan-t5-base
License Apache-2.0

TokForge

Runs on-device in the TokForge app.

Use it in TokForge

  1. Install TokForge from Google Play, or join the iOS TestFlight.
  2. On Android, open Settings > Speed & engine > Models & Modules and download flan-t5-base Summarizer. TokForge can also fetch it by itself with its other helper files: when you attach a document, or once you allow automatic helper downloads.
  3. Once it is installed, TokForge sets it as the default summarizer. When TokForge answers a question about your attached documents by combining several passages, it can write that answer with this model, so the chat model is not tied up. It runs on the phone.
  4. On iOS, it is listed as FLAN-T5 Base Summarizer (MNN).

Collections: TokForge Voice and Helper Models.

Files

File Size What it is
config.json 1.40 KB T5 model config from google/flan-t5-base
decoder.mnn 173 MB Decoder for the first output token (4-bit)
decoder_with_past.mnn 166 MB Decoder for later tokens, with cache (4-bit)
encoder.mnn 145 MB Encoder graph and weights (4-bit)
generation_config.json 147 B Generation defaults
llm_config.json 934 B TokForge seq2seq settings (files, quantization, input prefix)
special_tokens_map.json 2.20 KB Special tokens
spiece.model 792 KB SentencePiece tokenizer model
tokenizer.json 2.42 MB Tokenizer
tokenizer_config.json 2.54 KB Tokenizer settings

The model is split the usual way for encoder-decoder inference: the encoder reads the input once, decoder.mnn writes the first output token and decoder_with_past.mnn writes the rest, reusing its cache.

About the model

FLAN-T5 base is Google's instruction-tuned T5 encoder-decoder model (about 250M parameters). TokForge gives it text with the summarize: prefix, as set in llm_config.json.

Architecture T5 encoder-decoder (12 encoder and 12 decoder layers, 768 hidden, 12 heads)
Quantization 4-bit weights, block size 128, asymmetric
Vocab 32,128 tokens (SentencePiece)
Input length Up to 512 tokens

How it was made

Exported as three ONNX graphs (encoder, decoder, and decoder with past), then each converted with MNN's converter to 4-bit asymmetric weights with block size 128 (llm_config.json records q4-block128-asymmetric).

The tokenizer files (spiece.model, tokenizer.json, tokenizer_config.json, special_tokens_map.json) and config.json / generation_config.json come from google/flan-t5-base.

Limitations

  • Built for TokForge's summarizer route; it is not a chat model
  • Inputs are limited to 512 tokens
  • Quantization (Q4) may slightly reduce quality compared to the full-precision original

License

FLAN-T5 base is released by Google under the Apache-2.0 license.

Credits

  • FLAN-T5 base by Google (Hugging Face). Used under the Apache-2.0 license.
  • Converted to MNN with MNN by Alibaba (Apache-2.0).
Downloads last month
995
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for darkmaniac7/flan-t5-base-MNN

Quantized
(22)
this model

Collection including darkmaniac7/flan-t5-base-MNN