MIST-Extract-1.0

Highlights

MIST-Extract-1.0 turns a document into JSON. Give it a JSON template and a document (text, a photo or scan, or both), and it fills in the template. Each field it returns comes with a calibrated confidence. It is a 4B model, built for English and African-language documents. For a faster 2B version, see MIST-Extract-Flash-1.0.

• English documents, extracted end to end: on 486 held-out English documents (receipts, notices, emails, contracts and more), it gets 91.8% of documents completely right. NuExtract3, built on the same 4B base, gets 67.9%. On English receipt photos it gets every field right.
• Strongest model we tested on African languages. On named-entity extraction in 21 African languages (MasakhaNER 2.0), it scores 0.728 field accuracy. NuExtract3 scores 0.393, Qwen3.5-4B 0.432 and GLiNER2.5-multi 0.452.
• Ahead of NuExtract3 on every part of our held-out set, general documents included (0.941 vs 0.922), at the same size and slightly faster (6.0 vs 5.6 documents per second).
• Calibrated confidence for every field. When it says 0.66, it is right about 66% of the time (calibration error 0.008 on held-out fields), so you can send uncertain values to a person and keep the rest automatic.
• Same template format as NuExtract, so templates move between the two without changes.

What it's for

MIST-Extract is built for the everyday documents of African businesses and their customers, in English and African languages:

• Mobile money and banking: transfer confirmations, bank and wallet SMS, alerts and statements.
• Receipts and bills: shop, market, utility and prepaid receipts in naira, cedis, shillings, rand and other African currencies, as text, photos or screenshots.
• Letters and forms: tenancy and school letters, notices, IDs, tickets, complaints and claims.
• Names in 20+ African languages: people, organisations, places and dates.

On these documents it is more accurate than the general-purpose extractors we tested, and its confidence tells you which fields to check. For general Western invoices, multi-page PDFs and foreign receipts, use a general-purpose extractor such as NuExtract3.

Model Overview

Base	Qwen/Qwen3.5-4B-Base (Apache-2.0), full fine-tune, vision encoder frozen
Input	a JSON template, a document as text and/or one image, optional instructions
Output	JSON with the template's shape, plus a confidence for each field
Precision	bf16
Context	262,144-token window from the Qwen3.5 base; we serve it at 131,072. Images are scaled to about 1.3 megapixels
Smaller sibling	MIST-Extract-Flash-1.0 (2B), 1.6× faster

Template format

The template has the shape of the answer. Each leaf names a type:

Type	Meaning
"verbatim-string"	copied exactly from the document
"string"	free text, may be normalised or paraphrased
"number", "integer"	plain numbers, no currency symbols or thousands separators
"date"	ISO YYYY-MM-DD
"date-time"	ISO date and time
"boolean"	true or false
["a", "b"]	pick one option
[["a", "b"]]	pick any number of options
["string"]	a list of values
[{...}]	a list of objects

A missing value comes back as null, and an empty list as [].

Quickstart

pip install "transformers>=5.17" torch pillow huggingface_hub

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("olaverse/MIST-Extract-1.0")
sys.path.insert(0, path)                      # the repo ships mist_extract.py
from mist_extract import Extractor

x = Extractor.from_pretrained(path, device="cuda")

template = {
    "merchant": "verbatim-string",
    "date": "date",
    "total": "number",
    "payment_method": ["cash", "card", "mobile money", "bank transfer"],
    "items": [{"name": "verbatim-string", "quantity": "integer", "price": "number"}],
}

# from a photo or scan
out = x.extract(template, image="receipt.jpg")

# or from text
out = x.extract(template, text="Paid 2,500 naira at Mama Put on 3 May 2024 with my card. 2 plates of jollof.")

print(out["data"])         # {"merchant": "Mama Put", "date": "2024-05-03", "total": 2500, ...}
print(out["confidence"])   # same shape, a number in [0, 1] for each field

For many documents, x.extract_batch([{"template": ..., "text": ..., "image": ...}, ...]) runs them in one batch. Add instructions="..." to steer formats or definitions, for example "amounts in kobo".

Offline batch inference (vLLM)

The model is a standard Qwen3_5ForConditionalGeneration checkpoint, so vLLM serves it as is. Build the prompt below, pass the image (if any) as multi_modal_data, and decode greedily. Field confidences need token probabilities, which the Extractor above computes for you.

import json
from PIL import Image
from vllm import LLM, SamplingParams

SYSTEM = "You extract information from documents into JSON that matches the template."

def prompt(template, text="", instructions="", image=False):
    parts = [f"# Template:\n{json.dumps(template, ensure_ascii=False)}"]
    if instructions:
        parts.append(f"# Instructions:\n{instructions}")
    doc = ("<|vision_start|><|image_pad|><|vision_end|>\n" if image else "") + text
    parts.append(f"# Document:\n{doc}")
    return (f"<|im_start|>system\n{SYSTEM}<|im_end|>\n<|im_start|>user\n" + "\n".join(parts) +
            "<|im_end|>\n<|im_start|>assistant\n")

def fit(img, max_tokens=1280):
    """Downscale to at most 1,280 vision tokens (one token per 32 x 32 pixels), as in training."""
    w, h = img.size
    if w * h > max_tokens * 1024:
        s = (max_tokens * 1024 / (w * h)) ** 0.5 * 0.97
        img = img.resize((int(w * s), int(h * s)))
    return img

def main():
    llm = LLM("olaverse/MIST-Extract-1.0", max_model_len=131072, limit_mm_per_prompt={"image": 1})
    params = SamplingParams(temperature=0, max_tokens=4096)
    template = {"merchant": "verbatim-string", "date": "date", "total": "number"}
    requests = [
        {"prompt": prompt(template, text="Paid 2,500 naira at Mama Put on 3 May 2024.")},
        {"prompt": prompt(template, image=True),
         "multi_modal_data": {"image": fit(Image.open("receipt.jpg").convert("RGB"))}},
    ]
    for out in llm.generate(requests, params):      # one batched call for any number of documents
        print(json.loads(out.outputs[0].text))


if __name__ == "__main__":                          # vLLM starts worker processes; the guard is required
    main()

OpenAI-compatible server (vLLM)

vllm serve olaverse/MIST-Extract-1.0 \
  --limit-mm-per-prompt '{"image": 1, "video": 0}' \
  --chat-template-content-format openai \
  --max-model-len 131072

The model's chat template builds the extraction prompt. Pass the template (and optional instructions) as chat_template_kwargs, and put the document in the user message: text, one image, or both. Send the template as compact JSON, as in training, and use temperature=0. Large photos are downscaled automatically to the training size (about 1.3 megapixels).

Text

import json
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

template = {
    "store": "verbatim-string",
    "date": "date",
    "total": "number",
    "currency": ["NGN", "GHS", "KES", "ZAR", "USD", "Other"],
    "items": [{"name": "verbatim-string", "quantity": "integer", "price": "number"}],
}

response = client.chat.completions.create(
    model="olaverse/MIST-Extract-1.0",
    temperature=0,
    messages=[{
        "role": "user",
        "content": [{"type": "text", "text": "Yesterday (12 March 2025) I paid ₦7,500 at Mama Put Kitchen, Yaba: "
                     "2 plates of jollof rice at ₦3,000 each and a bottle of Zobo for ₦1,500."}],
    }],
    extra_body={"chat_template_kwargs": {"template": json.dumps(template)}},
)
print(response.choices[0].message.content)

Output:

{"store": "Mama Put Kitchen, Yaba", "date": "2025-03-12", "total": 7500, "currency": "NGN",
 "items": [{"name": "jollof rice", "quantity": 2, "price": 3000}, {"name": "Zobo", "quantity": 1, "price": 1500}]}

Image

import base64, json
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

with open("receipt.jpg", "rb") as f:
    data_url = "data:image/jpeg;base64," + base64.b64encode(f.read()).decode("utf-8")

template = {"store": "verbatim-string", "date": "date", "total": "number", "tax": "number"}

response = client.chat.completions.create(
    model="olaverse/MIST-Extract-1.0",
    temperature=0,
    messages=[{"role": "user", "content": [{"type": "image_url", "image_url": {"url": data_url}}]}],
    extra_body={"chat_template_kwargs": {"template": json.dumps(template)}},
)
print(response.choices[0].message.content)

Instructions

response = client.chat.completions.create(
    model="olaverse/MIST-Extract-1.0",
    temperature=0,
    messages=[{"role": "user", "content": "Your MTN MoMo transfer of GHS 250.00 to KOFI MENSAH was successful. "
                                          "Fee GHS 2.50. Ref 88213374. 14/10/2025"}],
    extra_body={"chat_template_kwargs": {
        "template": json.dumps({"recipient": "verbatim-string", "amount": "number", "fee": "number",
                                "reference": "verbatim-string", "date": "date"}),
        "instructions": "Dates in this message are day/month/year.",
    }},
)
print(response.choices[0].message.content)
# {"recipient": "KOFI MENSAH", "amount": 250, "fee": 2.5, "reference": "88213374", "date": "2025-10-14"}

The server returns the JSON only. For per-field confidences, use the Extractor from the Quickstart.

Confidence

A field's confidence is the lowest probability among the tokens that spell its value, put through a calibration (Platt scaling) fitted on held-out fields and stored in mist_extract_config.json. On the held-out half not used for fitting, calibration error (ECE) falls from 0.045 to 0.008:

Calibrated confidence	Fields	Share actually right
below 0.5	356	40%
0.5 to 0.8	694	66%
0.8 to 0.95	916	89%
0.95 to 0.99	1,917	98%
0.99 and above	2,859	99.9%

Route fields below a threshold you choose to review. An empty list ([]) has no value tokens, so it has no confidence.

Evaluation

Every model saw the same documents (text, or the same image), the same template and instructions, greedy decoding and up to 2,048 new tokens, in the prompt format its makers document. All answers are scored by the same rules:

• Facts are scored exactly (numbers, dates, IDs, choices), after parsing numbers and dates written in any format.
• Free text is scored by word overlap, with a Jaccard index of at least 0.5.
• Lists are compared as sets.

Field accuracy is the share of fields right. Documents fully right is the share of documents with every field right.

Benchmark	Documents	What it is
Olaverse held-out	2,016	Held out from our training data, so in-domain: receipt photos, African documents, general documents, and MasakhaNER 2.0 test passages (about 50 per language). 32 languages.
CORD-v2 test	100	Indonesian receipt photos. We trained on CORD's training split.
nanonets KIE	987	Malaysian receipt photos, not used in training.
Invoice Extraction Benchmark v1	181	Invoices: PDFs (given as text), phone photos and scans. Not used in training.

Models compared: MIST-Extract-1.0 (4B), MIST-Extract-Flash-1.0 (2B), NuExtract3 (4B), Qwen3.5-4B without fine-tuning, LFM2-1.2B-Extract for text with LFM2.5-VL-1.6B-Extract for images, and GLiNER2.5-multi. GLiNER2.5-multi extracts spans only, so it is scored on the named-entity lists only.

English documents (Olaverse held-out, 486 documents)

	MIST-Extract (4B)	MIST-Extract-Flash (2B)	NuExtract3 (4B)	Qwen3.5-4B	LFM2-Extract
Documents fully right	91.8%	90.5%	67.9%	34.0%	4.5%
Field accuracy, receipt photos (327)	1.000	0.999	0.925	0.738	0.565
Field accuracy, text documents (159)	0.965	0.956	0.967	0.940	0.700

Our focus: field accuracy

	MIST-Extract (4B)	MIST-Extract-Flash (2B)	NuExtract3 (4B)	Qwen3.5-4B	LFM2-Extract	GLiNER2.5-multi
Olaverse held-out (2,016 documents)	0.898	0.879	0.801	0.762	0.432	—
↳ MasakhaNER 2.0, 21 African languages	0.728	0.701	0.393	0.432	0.003	0.452
↳ receipt photos	1.000	0.999	0.925	0.738	0.565	—
↳ African documents	0.931	0.912	0.919	0.890	0.556	—
↳ general documents	0.941	0.900	0.922	0.912	0.553	—
CORD-v2 receipts (we trained on its train split)	0.899	0.921	0.672	0.619	0.282	—

The fields that matter most

Dates, amounts and currencies on the Olaverse held-out documents (receipts, mobile money, bills, letters, forms):

Field type	MIST-Extract (4B)	MIST-Extract-Flash (2B)	NuExtract3 (4B)	Qwen3.5-4B
Dates (792 fields)	0.979	0.975	0.884	0.833
Amounts and numbers (1,757)	0.984	0.970	0.964	0.919
Currency (119)	0.975	0.975	0.966	0.403

Documents fully right

	MIST-Extract (4B)	MIST-Extract-Flash (2B)	NuExtract3 (4B)	Qwen3.5-4B	LFM2-Extract
Olaverse held-out	62.4%	57.6%	37.3%	29.4%	1.4%
CORD-v2 receipts	64.0%	71.0%	30.0%	25.0%	5.0%

Outside our focus: general Western receipts and invoices

These are not the documents MIST-Extract was built for, and a general-purpose extractor does better on them. They are reported for completeness:

	MIST-Extract (4B)	MIST-Extract-Flash (2B)	NuExtract3 (4B)	Qwen3.5-4B	LFM2-Extract
nanonets KIE (987 Malaysian receipt photos), field accuracy	0.819	0.759	0.917	0.899	0.654
Invoice Extraction Benchmark (181 invoices), field accuracy	0.793	0.759	0.907	0.880	0.451

Valid JSON

Benchmark	MIST-Extract (4B)	MIST-Extract-Flash	NuExtract3	Qwen3.5-4B	LFM2-Extract
Olaverse held-out	100%	100%	100%	99.9%	94.1%
Invoice Extraction Benchmark (outside our focus)	94.5%	96.1%	96.7%	95.6%	80.7%

Speed

Documents per second on one RTX A6000 (48 GB) under vLLM 0.31, greedy decoding, all 3,284 benchmark documents (GLiNER2.5-multi: its 1,000 entity documents).

MIST-Extract (4B)	MIST-Extract-Flash	NuExtract3	Qwen3.5-4B	LFM2-Extract	GLiNER2.5-multi
6.0	9.7	5.6	5.4	7.7	18.4

Supported languages

The template keys stay in English; the document can be in any of the languages below. Field accuracy is on the Olaverse held-out set, by document language.

• Full support: documents of every kind, many training examples. English, Naija (Nigerian Pidgin), isiZulu, Kiswahili, Hausa, Yorùbá, Igbo, Somali and Amharic.
• Named entities: names of people, organisations, places and dates, from MasakhaNER 2.0 (600 training passages each). Twi, chiShona, Fon, Chichewa, Bambara, Ghomálá', Mooré, Setswana, Kinyarwanda, Wolof, Ewe, isiXhosa, Luganda and Luo. Other document types in these languages work less reliably.
• Light coverage: Indonesian (CORD receipts), Chinese, Arabic, Hindi, Russian, Spanish, French, German, Turkish and Portuguese, with 15 to 900 training examples each. These come largely from the Qwen3.5 base; the test sets are too small to report (1 to 6 documents).

Language	Documents	MIST-Extract (4B)	MIST-Extract-Flash	NuExtract3	Qwen3.5-4B	GLiNER2.5-multi
English	486	0.985	0.981	0.943	0.824	—
Naija (Nigerian Pidgin)	111	0.920	0.908	0.885	0.844	0.420
isiZulu	113	0.901	0.850	0.810	0.777	0.541
Kiswahili	109	0.889	0.864	0.845	0.818	0.438
Hausa	109	0.866	0.851	0.794	0.786	0.293
Yorùbá	107	0.866	0.829	0.751	0.757	0.594
Igbo	106	0.899	0.866	0.789	0.787	0.427
Somali	78	0.929	0.923	0.931	0.910	—
Amharic	75	0.893	0.859	0.870	0.846	—
Twi	50	0.839	0.863	0.645	0.677	0.645
chiShona	50	0.835	0.795	0.291	0.276	0.339
Fon	50	0.800	0.791	0.443	0.513	0.513
Chichewa	50	0.770	0.746	0.418	0.459	0.451
Kinyarwanda	50	0.750	0.692	0.433	0.375	0.300
Bambara	50	0.748	0.732	0.386	0.512	0.512
Mooré	50	0.740	0.707	0.431	0.593	0.577
Ghomálá'	50	0.726	0.726	0.425	0.540	0.637
Wolof	50	0.720	0.678	0.347	0.432	0.432
Setswana	50	0.698	0.713	0.450	0.457	0.465
Luganda	50	0.667	0.576	0.356	0.348	0.402
Ewe	50	0.658	0.650	0.385	0.410	0.325
isiXhosa	50	0.650	0.585	0.260	0.333	0.431
Luo	50	0.610	0.542	0.195	0.203	0.297

GLiNER2.5-multi is scored on the named-entity lists only. Entity lists are scored strictly: a list counts as right only if it holds exactly the right names.

Training

• Method: full fine-tune of Qwen3.5-4B-Base for 2 epochs on 44,145 examples, 16,573 of them with an image. The loss covered only the answer, and the vision encoder was frozen. Learning rate 1.5e-5, 8-bit AdamW, effective batch 16, bf16, on one A100 80 GB.
• Public data:
  • MasakhaNER 2.0 (AFL-3.0): named entities in 20 African languages, capped at 600 passages per language.
  • CORD-v2 (CC-BY-4.0): 900 receipt photos.
• Rendered receipts and invoices: 12,000 documents in African currencies and layouts. The renderer knows every printed value, so the answers are exact.
• Generated documents: African documents (mobile-money and bank SMS, utility bills, IDs and forms, tenancy and school letters, tickets) and general documents (emails, contracts, news, reports, listings), with phone-screenshot versions. Each answer was written by one model and checked by a second, and fields the two disagreed on were removed.
• Templates: key names vary and fields are randomly sub-selected, so the model follows the template rather than memorising one.

Limitations

• Scope. Built for African and English everyday documents (see What it's for). On general Western invoices and foreign receipts, dates, decimal amounts and currency codes go wrong more often; use a general-purpose extractor for those.
• One image per call. For a multi-page scan, send the pages as text or extract page by page.
• Entity lists are copied as written, so spelling variants of a name count as different values.

Citation

@misc{mist-extract-2026,
  title  = {MIST-Extract: document extraction for English and African languages, with field confidence},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/MIST-Extract-1.0}
}