nanash66/typhoon-ocr1.5-2b-NVFP4

Typhoon OCR 1.5 2B — NVFP4 Quantized

NVFP4 (W4A4) quantized version of typhoon-ai/typhoon-ocr1.5-2b for NVIDIA Blackwell GPUs (GB10 / sm_121).

Model Details

Property	Value
Base Model	typhoon-ai/typhoon-ocr1.5-2b
Architecture	Qwen3VLForConditionalGeneration
Quantization	NVFP4 (W4A4), group_size=16
Quantizer	NVIDIA ModelOpt 0.45.0
Target GPU	NVIDIA Blackwell (GB10 / sm_121)
Quantized Size	2.23 GB
Context Length	262,144 tokens
Hidden Size	2048
Layers	28
Attention Heads	16 (KV: 8)
Vision	Qwen3VL Vision Encoder (1024 dim, 24 layers)

Quantization Configuration

• Algorithm: NVFP4 (NVIDIA Floating Point 4-bit)
• Weights: 4-bit float, group_size=16
• Activations: 4-bit float, group_size=16
• Excluded modules: lm_head, model.language_model.embed_tokens, model.visual*

Files

File	Size
model.safetensors	2.23 GB
config.json	Model config with quantization_config
hf_quant_config.json	HF quantization metadata
generation_config.json	Generation defaults
chat_template.jinja	Chat template
tokenizer.json	Tokenizer model
tokenizer_config.json	Tokenizer config
preprocessor_config.json	Vision preprocessor config
video_preprocessor_config.json	Video preprocessor config

Intended Uses & Limitations

This is a task-specific model intended to be used only with the provided prompt. It does not include any guardrails or VQA capability. Due to the nature of large language models (LLMs), a certain level of hallucination may occur. We recommend that developers carefully assess these risks in the context of their specific use case.

Important: This model is a single-prompt model — it works with one specific extraction prompt of Typhoon OCR v1.5 only. Other prompts will not produce correct results.

Prompting

The official extraction prompt for Typhoon OCR v1.5:

prompt = """Extract all text from the image.

Instructions:
- Only return the clean Markdown.
- Do not include any explanation or extra text.
- You must include all information on the page.

Formatting Rules:
- Tables: Render tables using <table>...</table> in clean HTML format.
- Equations: Render equations using LaTeX syntax with inline ($...$) and block ($$...$$).
- Images/Charts/Diagrams: Wrap any clearly defined visual areas (e.g. charts, diagrams, pictures) in:

<figure>
Describe the image's main elements (people, objects, text), note any contextual clues (place, event, culture), mention visible text and its meaning, provide deeper analysis when relevant (especially for financial charts, graphs, or documents), comment on style or architecture if relevant, then give a concise overall summary. Describe in Thai.
</figure>

- Page Numbers: Wrap page numbers in <page_number>...</page_number> (e.g., <page_number>14</page_number>).
- Checkboxes: Use ☐ for unchecked and ☑ for checked boxes."""

Usage

Recommended: vLLM + typhoon-ocr library

We recommend running inference via vLLM with the typhoon-ocr library rather than raw transformers:

pip install vllm typhoon-ocr

Serve the local NVFP4 model:

vllm serve nanash66/typhoon-ocr1.5-2b-NVFP4 \
  --quantization modelopt \
  --max-model-len 49152 \
  --served-model-name typhoon-ocr-nvfp4

from typhoon_ocr import ocr_document

markdown = ocr_document(
    "image.png",
    model="typhoon-ocr-nvfp4",
    figure_language="Thai",
    task_type="v1.5",
    base_url="http://localhost:8000/v1",
    api_key="no-key",
)
print(markdown)

Alternative: HuggingFace Transformers

Note: This model requires a Blackwell GPU (sm_121) with NVIDIA ModelOpt runtime for NVFP4 inference.

from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model = AutoModelForImageTextToText.from_pretrained(
    "nanash66/typhoon-ocr1.5-2b-NVFP4",
    dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("nanash66/typhoon-ocr1.5-2b-NVFP4")

# Load image
img = Image.open("image.png")

# IMPORTANT: The model was trained at a fixed image dimension of 1800px.
# Resize the image before inference.
def resize_if_needed(img, max_size=1800):
    if max(img.size) > max_size:
        scale = max_size / max(img.size)
        img = img.resize((int(img.width * scale), int(img.height * scale)))
    return img

img = resize_if_needed(img, 1800)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": img},
            {"type": "text", "text": prompt},
        ],
    }
]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

# Inference
generated_ids = model.generate(**inputs, max_new_tokens=10000)
generated_ids_trimmed = [
    out_ids[len(in_ids):]
    for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(output_text[0])

Derivative Of

This is a derivative work of typhoon-ai/typhoon-ocr1.5-2b, released under the Apache-2.0 license.

Citation

If you find this model useful, please cite the original works:

@misc{nonesung2026typhoonocropenvisionlanguage,
      title={Typhoon OCR: Open Vision-Language Model For Thai Document Extraction},
      author={Surapon Nonesung and Natapong Nitarach and Teetouch Jaknamon and Pittawat Taveekitworachai and Kunat Pipatanakul},
      year={2026},
      eprint={2601.14722},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2601.14722},
}

@misc{typhoon2,
      title={Typhoon 2},
      year={2024},
      eprint={2412.13702},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2412.13702},
}

@misc{thaiocrbench,
      title={ThaiOCRBench},
      year={2025},
      eprint={2511.04479},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2511.04479},
}

License

Apache-2.0 (same as base model)