Hugging Face's logo

typhoon-ai
/
typhoon-ocr2-2b

Image-Text-to-Text
Transformers
Safetensors
Thai
English
qwen3_5
ocr
document-parsing
layout-analysis
table-recognition
thai
conversational
Model card Files
xet
 Community

Typhoon OCR V2 2B

Typhoon OCR V2 benchmark results

Typhoon OCR V2 2B is a document parser for Thai and English. Given an image of a page, it returns the page's layout elements in reading order, each with a category, a bounding box and its text. Tables come out as HTML, formulas as LaTeX and other text as Markdown. Beyond parsing, the model is meant to answer general questions about a document (visual question answering), as well as classify documents, extract key information as JSON and read tables, formulas and charts on their own.

The model is a fine-tune of Infinity-Parser2-Flash. We trained it with supervised fine-tuning on curated Thai data and public English and Chinese data, then with GRPO, using rewards that give zero to outputs that run into the token limit. That second stage targets repetition loops on photographed pages. A smaller model, Typhoon OCR V2 0.8B, is released alongside this one.

Our goal was to start from a parser that is already strong on English, raise its Thai accuracy, and keep what it already knew about English. Infinity-Parser2-Flash reads English and Chinese well but is weaker on Thai: on MDPBench it scores 75.4 on English pages and 61.3 on Thai pages. After our training, the Thai score rose by 20 points, to 81.3, and the English score went up instead of dropping, to 78.4. On olmOCR-bench and OmniDocBench, the scores stayed about the same as the base model's (see Comparison with the base model).

Use of this model is subject to the OpenTyphoon Terms and Conditions.

Use vLLM 0.33.0 or newer. On older vLLM versions, run in eager mode (--enforce-eager for vllm serve, enforce_eager=True for LLM(...)). With vLLM 0.27.1 in its default compiled mode (torch.compile and CUDA graphs), we measured much lower accuracy: the same weights scored 70.8 on MDPBench in compiled mode and 80.0 in eager mode. With vLLM 0.33.0, the default mode scored 79.7 and eager mode 79.3, so the problem no longer appears there. Transformers is not affected. The benchmark scores on this card were measured with vLLM 0.27.1 in eager mode. See Usage.

Key enhancements over Typhoon OCR 1.5

  • Layout with bounding boxes. Each element comes with a category (13 types, such as title, text, table, formula, header and footer) and a box, in reading order. Typhoon OCR 1.5 returned the page as Markdown without positions.
  • General visual question answering. The model answers free-form questions about a document and also handles document classification, key-information extraction to JSON and chart-to-data tasks. On ThaiOCRBench, which covers these tasks, the score rises from 40.5 to 59.9.
  • Higher accuracy on public Thai and English benchmarks: MDPBench from 62.8 to 79.8 (Thai pages from 64.3 to 81.3), olmOCR-bench from 62.0 to 84.7, OmniDocBench v1.6 from 70.97 to 90.97, and SEA-DocBench NED from 0.1726 to 0.1153 (lower is better).
  • Better on photographed pages: MDPBench photographed pages from 61.6 to 79.7.
  • Markdown is still available. A few lines of code convert the JSON to Markdown, with or without headers and footers (see Usage).
  • A smaller 0.8B model for page parsing at lower cost.

Model details

Parameters 2B, BF16
Architecture Qwen3.5 (Qwen3_5ForConditionalGeneration)
Base model infly/Infinity-Parser2-Flash
Training Supervised fine-tuning, then GRPO with a LoRA adapter, merged into the weights in fp32 and stored in bf16
Languages Thai and English. The training data also contains Chinese, which we did not evaluate separately.
Input One page image. Trained with images of up to 4 MP; the benchmark numbers below use up to 16.7 MP.
Output A JSON list of layout elements in a json code block

Comparison with the base model

Both models were run with the same prompt, settings and scripts, with vLLM 0.27.1 in eager mode. Infinity-Parser2-Flash has the same architecture as our models and its scores also drop in vLLM 0.27.1's compiled mode (61.5 against 68.3 on MDPBench), so we ran it in eager mode too.

Benchmark Infinity-Parser2-Flash Typhoon OCR V2 2B Change
MDPBench, Thai pages ↑ 61.3 81.3 +20.0
MDPBench, English pages ↑ 75.4 78.4 +3.0
MDPBench, photographed pages ↑ 65.3 79.7 +14.4
MDPBench, digital pages ↑ 78.9 81.3 +2.4
MDPBench, overall ↑ 68.3 79.8 +11.5
ThaiOCRBench ↑ 35.0 59.9 +24.9
SEA-DocBench, Thai NED ↓ 0.1268 0.1059 −0.0209 (better)
SEA-DocBench, English NED ↓ 0.1264 0.1248 −0.0016 (better)
olmOCR-bench ↑ 84.4 84.7 +0.3
OmniDocBench v1.6 ↑ 91.03 90.97 −0.06

Thai accuracy rose on all three benchmarks with Thai pages. On olmOCR-bench (English) and OmniDocBench (English and Chinese) the scores barely changed: olmOCR-bench went up by 0.3 points (its 95% confidence interval is about ±0.8), and OmniDocBench went down by 0.06 points. Within OmniDocBench, tables improved (TEDS 81.76 to 82.94) and formulas got worse (CDM 95.02 to 93.95).

Results

We ran every model below ourselves with the same evaluation scripts. The numbers are not copied from other papers or model cards, so some differ from what the other authors report.

  • Typhoon OCR V2 and Infinity-Parser2-Flash use the doc2json prompt shown in Usage, images of up to 16.7 MP, greedy decoding, at most 16,384 new tokens and vLLM 0.27.1 in eager mode. Their JSON is converted to Markdown with Infinity-Parser2's conversion code, which drops headers, footers and page footnotes. The model still outputs these elements; only the conversion leaves them out. Usage shows how to keep them.
  • The other models use the prompts and post-processing published with them.
  • MDPBench and SEA-DocBench are limited to their English and Thai subsets.
  • PaddleOCR-VL-1.6 does not support VQA, so it has no ThaiOCRBench score.

In each column, bold is the best result and underlined the second best. ↑ means higher is better and ↓ lower is better.

Model Params MDPBench ↑ MDPBench TH ↑ olmOCR-bench ↑ ThaiOCRBench ↑ SEA-DocBench NED ↓ OmniDocBench v1.6 ↑
Chandra-OCR-2 5B 74.2 71.1 84.4 26.4 0.1141 89.65
dots.mocr 3B 76.2 73.6 85.2 37.9 0.1135 91.09
PaddleOCR-VL-1.6 0.9B 72.5 68.4 80.8 n/a 0.1323 95.83
Infinity-Parser2-Flash 2B 68.3 61.3 84.4 35.0 0.1266 91.03
Typhoon OCR 1.5 2B 62.8 64.3 62.0 40.5 0.1726 70.97
Typhoon OCR V2 0.8B 0.8B 77.4 77.2 81.7 36.4 0.1081 89.37
Typhoon OCR V2 2B 2B 79.8 81.3 84.7 59.9 0.1153 90.97

Per-benchmark component scores will be in the technical report (coming soon).

Examples

Examples 1 to 6 are outputs of this model on benchmark items, run with vLLM 0.27.1 in eager mode. The two parsing examples use max_pixels at 4 MP, the image size the model was trained with; examples 3 to 6 come from our ThaiOCRBench run, where each item's question is the prompt. Examples 7 and 8 use images from our training data and were run with vLLM 0.33.0, so they show the output format rather than accuracy on new documents.

1. Full-page parsing: a dense Thai exam page

Thai exam page and predicted layout boxes

Left: the input (MDPBench). Right: the predicted boxes, numbered in reading order. 19 of the 29 predicted boxes match the benchmark's layout boxes (IoU ≥ 0.5). Most of the others are not misplaced: the model gives each line of the numbered conclusions (items 20–29) its own box, while the benchmark puts each pair of lines in one box.

Model output (first 5 of 29 elements) 
[{"bbox": [940, 14, 977, 26], "category": "header", "text": "8 / 29"},
 {"bbox": [409, 75, 590, 92], "category": "title", "text": "การสรุปความจากภาษา"},
 {"bbox": [129, 97, 870, 169], "category": "text", "text": "คำสั่ง ในแต่ละข้อจะประกอบด้วยเงื่อนไขหรือข้อมูลที่กำหนดให้ และข้อสรุปเป็นคู่ ๆ จากเงื่อนไขนั้นให้ศึกษาเงื่อนไขที่กำหนดให้ก่อน แล้วจึงอาศัยความรู้ที่ได้จากเงื่อนไขดังกล่าวมาใช้พิจารณาข้อสรุปทั้งสองของแต่ละข้อ แล้วทำตอบลงในกระดาษคำตอบ โดยยึดหลักในการทำตอบดังนี้"},
 {"bbox": [130, 202, 272, 219], "category": "title", "text": "หลักในการทำตอบ"},
 {"bbox": [129, 224, 474, 245], "category": "text", "text": "ตอบ 1. ถ้าข้อสรุปทั้งสองเป็นจริงตามเงื่อนไข"},
 ...]

2. Full-page parsing: a Thai nurse-training handout with references

Thai training handout and predicted layout boxes

18 of the 20 predicted boxes match the benchmark's layout boxes (IoU ≥ 0.5).

Model output (first 5 of 20 elements) 
[{"bbox": [497, 43, 883, 77], "category": "header", "text": "เอกสารประกอบการอบรม การดูแลผู้ป่วยบาดเจ็บขั้นสูง รุ่นที่ 13\nAdvanced trauma emergency care for nurse :ATEN"},
 {"bbox": [897, 45, 929, 61], "category": "header", "text": "127"},
 {"bbox": [177, 95, 723, 156], "category": "text", "text": "สูตรการคำนวณพลังงานแคลอรี่ (25 Kcal x kg.)+(40 Kcal x % BSA)\nตัวอย่างชายไทย นน. 50 กก. Burn 80% TBSA Total Calories/day\n= (25x50)+(40x80)=4450 Kcal/day"},
 {"bbox": [179, 182, 362, 198], "category": "title", "text": "10.การพยาบาลด้านจิตใจ"},
 {"bbox": [208, 203, 460, 220], "category": "text", "text": "■ มีการประคับประคองด้านจิตใจ"},
 ...]

3. Key-information extraction: ingredients on a snack bag

Thai snack bag label

Prompt:

จากรูปภาพกรุณาดึงข้อมูล  เปอร์เซ็นต์ของส่วนประกอบแต่ละอย่าง จากรูปภาพเป็นรูปแบบ JSON เป็นภาษาไทย เฉพาะอันที่มีเปอร์เซ็นต์

โดยมี key ตามนี้
มะพร้าว
น้ำตาล
เกลือสมุทร
กลิ่นคาราเมล
ตอบเฉพาะข้อความตามภาพตรง ๆ อย่างกระชับ โดยไม่ใส่คำอธิบายเพิ่มเติม

(Extract the percentage of each ingredient as JSON in Thai, using the keys coconut, sugar, sea salt and caramel flavour.)

Output, score 1.00 (every other open model we tested, including Typhoon OCR V2 0.8B: 0.00):

{"มะพร้าว": "68%", "น้ำตาล": "29%", "เกลือสมุทร": "2%", "กลิ่นคาราเมล": "1%"}

4. Key-information mapping: a PM2.5 advisory table

Thai PM2.5 advisory table

Prompt:

ตามข้อมูลในภาพกรุณาจับคู่รายการระดับความเข้มข้น PM2.5 (จำนวนจำกัด) เกี่ยวข้องดังต่อไปนี้: 'ดีมาก', 'ดี', 'ปานกลาง', 'เริ่มมีผลกระทบต่อสุขภาพ' และ 'มีผลกระทบต่อสุขภาพ' กับค่าที่จะถูกจับคู่ได้แก่ '7 จังหวัด', '0 จังหวัด', '28 จังหวัด', '16 จังหวัด' และ '25 จังหวัด' เลือกรายละเอียดจากเนื้อหาในเอกสารแล้วให้คำตอบกลับมาในรูปแบบ JSON โดยตรง:
ตอบเฉพาะข้อความตามภาพตรง ๆ อย่างกระชับ โดยไม่ใส่คำอธิบายเพิ่มเติม

(Match each PM2.5 level, from "very good" to "affecting health", to the number of provinces in the table, and answer in JSON.)

Output, score 1.00 (every other open model we tested, including Typhoon OCR V2 0.8B: 0.00):

{"ดีมาก": "28 จังหวัด", "ดี": "16 จังหวัด", "ปานกลาง": "25 จังหวัด", "เริ่มมีผลกระทบต่อสุขภาพ": "7 จังหวัด", "มีผลกระทบต่อสุขภาพ": "0 จังหวัด"}

5. Handwriting: lecture notes

Handwritten Thai lecture notes

Prompt: ข้อที่ 4 ในภาพนี้ เป็นข้อความที่เขียนด้วยมือว่าอะไรบ้าง ตอบเฉพาะข้อความตามภาพตรง ๆ อย่างกระชับ โดยไม่ใส่คำอธิบายเพิ่มเติม (What is handwritten in item 4?)

Output, score 0.96 (best of the four other open models we ran on ThaiOCRBench: 0.38; Typhoon OCR V2 0.8B: 0.84):

④ ผูกขาด // ผู้ขายรายเดียว ไม่มีสินค้าอื่นมาทดแทน ex. ไฟฟ้า ประปา

6. Text in a given region: a street sign

Bangkok street scene with the queried region marked

Prompt: แบ่งความยาวและความสูงของรูปภาพออกเป็น 1000 ส่วน แล้วช่วยดึงข้อความที่อยู่ในพิกัด [786, 615, 931, 675] ของรูปภาพออกมาให้หน่อย ตอบเฉพาะข้อความตามภาพตรง ๆ อย่างกระชับ โดยไม่ใส่คำอธิบายเพิ่มเติม (Read the text at [786, 615, 931, 675] on a 0–1000 grid; the box is drawn on the image.)

Output, score 1.00 (best of the four other open models we ran on ThaiOCRBench: 0.12; Typhoon OCR V2 0.8B: 0.16):

ชุมชนเสนานิคม 2

7. Extracting everything as JSON: a restaurant bill

Thai restaurant bill with the staff name blacked out

Without a list of fields, the model finds the fields itself. Prompt:

Perform open-schema key information extraction (WYSIWYG) on this image: one JSON object, printed labels as keys, values copied exactly, "" for blank fields, arrays for repeated rows. Return only JSON.

Output (we blacked out the staff name before running the model, and it returned "" for that field):

{
  "qr_code_instruction": "KSP03 สามารถสแกนด้วย",
  "restaurant_name": "Khao-So-i ข้าวโซอิ",
  "restaurant_address": "Khao-So-i ข้าว-โซ-อิ (เซ็นทรัล พระราม9)\nบริษัท ข้าว โซ อิ กรุ๊ป จำกัด สำนักงานใหญ่",
  "No. Branch": "00005",
  "address": "475 อาคารสิริภิญโญ ชั้น 17 ห้องเลขที่\n1701-1702 ถนนศรีอยุธยา แขวงถนนพญาไท\nเขตราชเทวี กรุงเทพมหานคร 10400",
  "TAX ID": "0105566077543",
  "Tel": "0611767121",
  "document_type": "Invoice",
  "Table": "A3",
  "Staff": "",
  "Guests": "1",
  "Date": "12/04/2026",
  "Time": "16:38",
  "items": [
    {
      "quantity": "1",
      "name": "ไส้ฮั่วอย่างโฮมเมด",
      "price": "139.00"
    },
    {
      "quantity": "1",
      "name": "ข้าวซอยเส้นสดเนื้อตุ๋น (S)",
      "price": "159.00"
    },
    {
      "quantity": "1",
      "name": "เผ็ดปวกติ",
      "price": "0.00"
    },
    {
      "quantity": "1",
      "name": "น้ำเปล่า (ขวด)",
      "price": "25.00"
    },
    {
      "quantity": "1",
      "name": "ไม่เอาน้ำแข็ง",
      "price": "0.00"
    }
  ],
  "Items": "3",
  "Subtotal": "323.00",
  "Service Charge(10%)": "32.30",
  "Rounding": "-0.30",
  "Total": "355.00",
  "Before VAT": "332.06",
  "VAT 7%": "23.24",
  "note": "\"หากต้องการออกใบกำกับภาษีเต็มรูปแบบ กรุณากรอกข้อมูลผ่าน QR Code ที่ระบุอยู่ด้านบนของหัวใบเสร็จ ภายใน 3 วันหลังวันที่ซื้อสินค้าเท่านั้น\"",
  "sign_off": "Thank you\nHave a good day!"
}

Most fields are correct. Two item names have small errors (ไส้ฮั่วอย่างโฮมเมด for ไส้อั่วย่างโฮมเมด, เผ็ดปวกติ for เผ็ดปกติ), the two options เผ็ดปกติ and ไม่เอาน้ำแข็ง are listed as separate items instead of under the dishes they belong to, and the first field comes from a line under the QR code that is too blurred to check.

8. Extracting everything as JSON: an infographic

Thai Rice Department infographic on certified rice seed

Same prompt as example 7. Output:

{
  "title": "ทำไมต้องใช้ เมล็ดพันธุ์ข้าว \"พันธุ์ดี\"??",
  "reasons": [
    "ตรงตามสายพันธุ์ และมีความบริสุทธิ์สูง",
    "ลดต้นทุนการผลิตข้าว",
    "เปอร์เซ็นต์ความงอกสูง ไม่ต่ำกว่าร้อยละ 80",
    "ให้ผลผลิตสูงกว่า เมล็ดพันธุ์ข้าวทั่วไป 10-20 %",
    "ผลผลิตมีคุณภาพดี",
    "ผ่านการทดสอบคุณภาพ ทุกขั้นตอนตามมาตรฐาน เมล็ดพันธุ์ข้าว",
    "ต้านทานโรคและแมลง"
  ],
  "หมายเหตุ": "สามารถติดต่อขอซื้อเมล็ดพันธุ์ข้าวพันธุ์ดีได้ที่ ศูนย์เมล็ดพันธุ์ข้าว/ศูนย์ข้าวชุมชน/ตัวแทนจำหน่ายเมล็ดพันธุ์ข้าวของกรมการข้าว ใกล้บ้านท่าน",
  "contact": {
    "website": "https://www.ricethailand.go.th/",
    "facebook": "กรมการข้าว : Rice news Channel",
    "tiktok": "Rice_loey",
    "telephone": "02-561-2866",
    "organization": "กองประชาสัมพันธ์ กรมการข้าว"
  }
}

Usage

The typhoon-ocr Python package (pip install typhoon-ocr), which runs Typhoon OCR 1.5, does not support V2 yet. Use vLLM or Transformers as shown below.

The model was trained on a fixed set of prompts. For full-page parsing, use this prompt exactly as written; it is the prompt from Infinity-Parser2.

PROMPT = """- Extract layout information from the provided PDF image.
- For each layout element, output its bbox, category, and the text content within the bbox.
- Bbox format: [x1, y1, x2, y2].
- Allowed layout categories: ['header', 'title', 'text', 'figure', 'table', 'formula', 'figure_caption', 'table_caption', 'formula_caption', 'figure_footnote', 'table_footnote', 'page_footnote', 'footer'].
- Text extraction and formatting:
  1) For 'figure', the text field must be an empty string.
  2) For 'formula', format text as LaTeX.
  3) For 'table', format text as HTML.
  4) For all other categories (e.g., text, title), format text as Markdown.
- The output text must be exactly the original text from the image, with no translation or rewriting.
- Sort all layout elements in human reading order.
- Final output must be a single JSON object."""

Although the last line asks for a JSON object, the model returns a JSON list inside a json code block:

[{"bbox": [62, 48, 938, 96], "category": "title", "text": "Annual Report 2025"},
 {"bbox": [62, 120, 938, 410], "category": "text", "text": "The company ..."},
 {"bbox": [62, 432, 938, 700], "category": "table", "text": "<table>...</table>"},
 {"bbox": [480, 950, 520, 968], "category": "footer", "text": "12"}]

Bounding boxes use a 0–1000 grid relative to the page width and height, so they do not depend on the image resolution. These helpers parse the output, convert it to Markdown and map a box back to pixels:

import json, re

def parse_elements(raw: str) -> list[dict]:
    """Parse the model output: a JSON list of elements inside a ```json fence."""
    m = re.search(r"```json\n(.*?)(?:\n```|$)", raw, re.DOTALL)
    text = (m.group(1) if m else raw).strip()
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        # Output stopped at the token limit: drop the last, incomplete element.
        cut = text.rfind('{"bbox":')
        return json.loads(text[:cut].rstrip().rstrip(",") + "]")

def to_markdown(elements: list[dict], keep_header_footer: bool = False) -> str:
    skip = set() if keep_header_footer else {"header", "footer", "page_footnote"}
    return "\n\n".join(e["text"] for e in elements if e.get("text") and e["category"] not in skip)

def to_pixels(bbox, width, height):
    x1, y1, x2, y2 = bbox
    return [x1 * width / 1000, y1 * height / 1000, x2 * width / 1000, y2 * height / 1000]

The JSON from the model always contains every element. to_markdown leaves out header, footer and page_footnote elements by default, which is how we evaluated the model. To keep them, pass keep_header_footer=True:

elements = parse_elements(raw)                           # raw: the text the model returned
md = to_markdown(elements)                               # drops header, footer, page_footnote (as in our evaluation)
md_all = to_markdown(elements, keep_header_footer=True)  # keeps every element

The examples below need PROMPT and the helpers above. They were tested with vLLM 0.33.0 and Transformers 5.15.1. Use the other settings in the code as they are; the benchmark numbers were produced with them.

pip install "vllm>=0.33.0"

On vLLM versions older than 0.33.0, add --enforce-eager to vllm serve, or enforce_eager=True to LLM(...). Without it, vLLM 0.27.1 compiles the model with torch.compile and CUDA graphs, and in that mode we measured about 9 points lower on MDPBench (70.8 against 80.0 for the same weights). We have not tested versions between 0.27.1 and 0.33.0.

vLLM server 

vllm serve typhoon-ai/typhoon-ocr2-2b \
  --served-model-name typhoon-ocr2 \
  --max-model-len 40960 \
  --limit-mm-per-prompt '{"image": 1}' \
  --mm-processor-kwargs '{"min_pixels": 2048, "max_pixels": 16777216}' \
  --default-chat-template-kwargs '{"enable_thinking": false}'
# vLLM < 0.33.0: add --enforce-eager

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

with open("page.jpg", "rb") as f:
    image_url = "data:image/jpeg;base64," + base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="typhoon-ocr2",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": image_url}},
        {"type": "text", "text": PROMPT},
    ]}],
    temperature=0.0,
    max_tokens=16384,
)
elements = parse_elements(resp.choices[0].message.content)
print(to_markdown(elements))

vLLM offline 

import base64
from vllm import LLM, SamplingParams

def main():
    llm = LLM(
        model="typhoon-ai/typhoon-ocr2-2b",
        max_model_len=40960,
        limit_mm_per_prompt={"image": 1},
        mm_processor_kwargs={"min_pixels": 2048, "max_pixels": 16777216},
        # enforce_eager=True,  # needed on vLLM < 0.33.0 (see above)
    )
    sampling = SamplingParams(temperature=0.0, max_tokens=16384)

    with open("page.jpg", "rb") as f:
        image_url = "data:image/jpeg;base64," + base64.b64encode(f.read()).decode()
    messages = [{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": image_url}},
        {"type": "text", "text": PROMPT},
    ]}]
    out = llm.chat(messages, sampling, chat_template_kwargs={"enable_thinking": False})
    print(to_markdown(parse_elements(out[0].outputs[0].text)))

if __name__ == "__main__":
    main()

Transformers 

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "typhoon-ai/typhoon-ocr2-2b"
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open("page.jpg").convert("RGB")
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": PROMPT},
]}]
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
    enable_thinking=False,
    processor_kwargs={"images_kwargs": {"min_pixels": 2048, "max_pixels": 16777216}},
).to(model.device)

out = model.generate(**inputs, max_new_tokens=16384, do_sample=False)
raw = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(to_markdown(parse_elements(raw)))

Questions about a document

To ask a question about a document, send the question as the text prompt in place of PROMPT. The answer comes back as plain text.

Extracting key information as JSON

To pull the fields out of a receipt, form, label or poster without listing them, use the prompt from example 7:

Perform open-schema key information extraction (WYSIWYG) on this image: one JSON object, printed labels as keys, values copied exactly, "" for blank fields, arrays for repeated rows. Return only JSON.

To get specific fields, list them in the prompt instead, as in example 3. We compared eight short prompts in Thai and English with the much longer prompt used in training, on 40 training documents. This prompt matched the training labels best among the short ones (F1 0.63, against 0.69 for the training prompt). Short prompts without the "" rule often filled a blank field with a nearby value.

Limitations

  • The model can fall into repetition loops, repeating text until it reaches the token limit. We saw this most on dense multi-column pages such as Thai newspapers. When that happens, parse_elements above keeps only the complete elements.
  • PaddleOCR-VL-1.6 is well ahead on tables and formulas (OmniDocBench table TEDS 93.38 against 82.94, formula CDM 97.48 against 93.95).
  • Key-information extraction is the weakest ThaiOCRBench task at 21.0, and the document-parsing task scores 38.1. Check the output of extraction tasks before using it downstream.
  • We did not hold out a validation set for parsing. The SFT checkpoint this model continues from was chosen by MDPBench score, so its MDPBench numbers are likely somewhat optimistic. The GRPO checkpoint is the last step of a fixed schedule and was not selected. The other four benchmarks were not used for any choice.
  • Bounding boxes and image size. The model was trained with images of up to 4 MP, and the Usage examples allow up to 16.7 MP. The boxes are about equally accurate at both settings: on 13 digital MDPBench pages larger than 4 MP, the share of predicted boxes matching the benchmark's layout boxes (F1 at IoU ≥ 0.5) was 0.72 at 4 MP and 0.74 at 16.7 MP. Text is more accurate at 16.7 MP: with max_pixels at 4,194,304 (4 MP), MDPBench drops from 79.8 to 78.2, mostly on Thai pages (81.3 to 78.8) and tables (TEDS 0.634 to 0.558). Keep 16.7 MP unless you need less GPU memory or faster inference. If your boxes look shifted, check your vLLM version first: in vLLM 0.27.1's compiled mode we saw boxes shifted to the right at 16.7 MP. Better box accuracy is a goal for the next version.
  • We studied Thai and English only. Chinese is in the training data and in OmniDocBench but was not evaluated on its own, and other languages were not tested.
  • Accuracy can depend on the inference engine. With vLLM 0.27.1, the default compiled mode lowered MDPBench by about 9 points in our tests; use vLLM 0.33.0 or newer, or eager mode on older versions (see Usage). Other serving engines (SGLang, llama.cpp, Ollama) have not been tested.
  • The model has no safety filtering and can output text that is not in the image. It is meant for reading documents, not for open-ended chat.

Citation

Typhoon OCR V2 technical report: coming soon.

Typhoon OCR 1.5:

@misc{nonesung2026typhoonocropenvisionlanguage,
  title         = {Typhoon OCR: Open Vision-Language Model For Thai Document Extraction},
  author        = {Surapon Nonesung and Natapong Nitarach and Teetouch Jaknamon and Pittawat Taveekitworachai and Kunat Pipatanakul},
  year          = {2026},
  eprint        = {2601.14722},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2601.14722}
}

Support

Downloads last month
102
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for typhoon-ai/typhoon-ocr2-2b

Base model

infly/Infinity-Parser2-Flash
Finetuned
(3)
this model
Quantizations
1 model

Space using typhoon-ai/typhoon-ocr2-2b 1

Collection including typhoon-ai/typhoon-ocr2-2b

Paper for typhoon-ai/typhoon-ocr2-2b