deAPI-ai/lightonocr-3-4b-q8

LightOnOCR-3-4B GGUF (Q8_0)

GGUF conversion of LightOnOCR-3-4B for llama.cpp. LightOnOCR-3-4B is an end-to-end OCR vision-language model that turns a document page image into Markdown, with tables as HTML.

Files

File	Contents	Precision	Size	SHA256
model.gguf	Language model	Q8_0	4.48 GB	39af92d0e22c262fdd1e611b0062c33961ad69c5166562172097c17cfdd4cfec
mmproj.gguf	Vision encoder and projector	F16	0.67 GB	486a4675f31a9b6d741c95e4605632c95d9b151c25eca22bcf1eddfee6cf901f

Original model

• Model: lightonai/LightOnOCR-3-4B
• Author: LightOn
• Architecture: Qwen3.5 vision-language (4B)
• Paper: LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
• Code: github.com/lightonai/LightOnOCR

Conversion

Converted from the official model.safetensors with llama.cpp's convert_hf_to_gguf.py:

• language model exported directly as Q8_0;
• vision encoder and projector exported as F16 (--mmproj);
• --no-mtp, because the upstream checkpoint ships without the multi-token-prediction head (mtp_num_hidden_layers: 0).

No other changes were made to the weights. The upstream chat template is embedded in model.gguf.

Usage with llama.cpp

llama-server -m model.gguf --mmproj mmproj.gguf --reasoning off

• --reasoning off is required. Without it llama-server enables the Qwen3.5 thinking mode, and the transcription ends up in reasoning_content while content is empty.
• Send the image only. As described in the upstream model card, transcription mode takes the page image with an empty prompt. Other instructions are out of distribution for the model.
• Image size: the upstream card recommends rendering PDF pages at 400 DPI and resizing so the longest side is 2048 px.

Example request body for /v1/chat/completions:

{
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "image_url", "image_url": { "url": "data:image/png;base64,..." } }
      ]
    }
  ],
  "temperature": 0
}

License

LightOnOCR-3-4B is released by LightOn under the Apache License 2.0, for research and commercial use. Its base architecture, Qwen3.5, is also released under the Apache License 2.0.

This repository redistributes the model under the same license. The files are a modified form of the original weights: they were converted to the GGUF format and the language model was quantized to Q8_0.

Citation

@misc{lightonocr2_2026,
  title        = {LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR},
  author       = {Said Taghadouini and Adrien Cavaill\`{e}s and Baptiste Aubertin},
  year         = {2026},
  howpublished = {\url{https://arxiv.org/abs/2601.14251}}
}

---

Hosted by deAPI-ai for use with GamerHash AI