| --- |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| base_model: |
| - baidu/Unlimited-OCR-2 |
| --- |
| |
| This is a randomly initialized tiny model based on [baidu/Unlimited-OCR-2](https://huggingface.co/baidu/Unlimited-OCR-2) for testing and debugging. |
|
|
| | Feature | Source | This repo | |
| |---|---|---| |
| | Checkpoint tensor elements | 3.34B BF16, 257 I64 | 9.04M BF16, 257 I64 | |
| | Safetensors weight files | 6.69 GB (2 files) | 18.12 MB (model.safetensors) | |
| | Decoder layers / hidden width / dense MLP | 12 / 1,280 / 6,848 | 4 / 32 / 256 | |
| | Q / KV heads / head dimension | 10 / 10 / 128 | 4 / 4 / 32 | |
| | Routed experts / top-k / shared experts | 64 / 6 / 2 | 16 / 6 / 2 | |
| | Expert intermediate width / initial dense layers | 896 / 1 | 64 / 1 | |
| | DASH index width / sparse block size | 128 / 32 | 32 / 32 | |
| | Page and token RoPE / sparse layer coverage | page2d / all layers | Same | |
| | SAM layers / width / global attention layers | 12 / 768 / 2,5,8,11 | 6 / 32 / 2,5 | |
| | CLIP layers / width | 24 / 1,024 | 4 / 32 | |
| | SAM neck / downsampling channels | 256 / 512,1,024 | 8 / 16,32 | |
| | Vision projector input / output | 2,048 / 1,280 | 64 / 32 | |
| | Global / crop image size / page visual tokens | 1,024 / 640 / 273 | Same | |
| | Vocabulary rows / tied embeddings | 129,280 / no | Same | |
| | Serialized precision | BF16 weights + I64 position IDs | Same | |
|
|
| Residual width 32 is independent of the four 32-dimensional attention heads: Q/K/V project to 128 channels and the output projection returns to the residual width. The included runtime exposes these config dimensions while preserving the upstream attention kernels. |
|
|
| | File path | Size | |
| |------|------| |
| | model.safetensors | 18.1MB | |
|
|
| ### Example usage |
|
|
| #### Transformers (✅ verified) |
|
|
| Install Transformers 4.52.1, `torchvision`, `Pillow`, `addict`, `easydict` and `einops` and load the included upstream remote code. The source registers only `mha_eager`; its OCR API runs DASH Triton sparse prefill, the reference masked sparse decode, and SDPA in SAM/CLIP. |
|
|
| ```python |
| def run_transformers(model_id: str): |
| import torch |
| from PIL import Image, ImageDraw |
| from transformers import AutoModel, AutoTokenizer |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
| model = AutoModel.from_pretrained(model_id, trust_remote_code=True, torch_dtype=torch.bfloat16).eval().cuda() |
| page = Image.new("RGB", (800, 1000), "white") |
| ImageDraw.Draw(page).text((50, 50), "Hello OCR. Page one.", fill="black") |
| text = model.infer(tokenizer, prompt="Read this document.", image_file=page, max_new_tokens=10) |
| print(text) |
| text, tokens = model.infer_multi(tokenizer, image_files=[page, page.copy()], max_new_tokens=10) |
| print(text, tokens) |
| |
| |
| if __name__ == "__main__": |
| run_transformers(model_id="tiny-random/unlimited-ocr-2") |
| ``` |
|
|
| #### SGLang (✅ verified) |
|
|
| Install Baidu's pinned `sglang-0.5.20.dev13+g75b9b06b02` wheel from the [source repo](https://huggingface.co/baidu/Unlimited-OCR-2/tree/c14071b549f4fcab10909f3c4acc6a2edb366947/wheel). The included external-model package reads the reduced vision config, separates residual width from attention head dimension, and gathers the TP-sharded DASH index projections before normalization. The upstream attention and routing kernels are retained. |
|
|
| ```bash |
| hf download tiny-random/unlimited-ocr-2 --local-dir ./unlimited-ocr-2 |
| pip install --no-deps ./unlimited-ocr-2 |
| |
| export SGLANG_EXTERNAL_MODEL_PACKAGE=unlimited_ocr_runtime |
| export SGLANG_UOCR_EANABLE_DOC_LIMIT=0 |
| python -m sglang.launch_server \ |
| --model-path ./unlimited-ocr-2 \ |
| --tp-size 4 \ |
| --trust-remote-code \ |
| --chat-template unlimited-ocr \ |
| --context-length 4096 \ |
| --max-total-tokens 8192 \ |
| --max-running-requests 4 \ |
| --cuda-graph-backend-decode full \ |
| --cuda-graph-max-bs-decode 4 \ |
| --cuda-graph-backend-prefill full \ |
| --cuda-graph-max-bs-prefill 2048 \ |
| --cuda-graph-bs-prefill 2048 \ |
| --max-prefill-tokens 2048 \ |
| --chunked-prefill-size 2048 \ |
| --cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 4}}' \ |
| --mem-fraction-static 0.3 \ |
| --host 0.0.0.0 --port 8000 |
| ``` |
|
|
| ```python |
| import base64 |
| from io import BytesIO |
| |
| import requests |
| from PIL import Image, ImageDraw |
| |
| page = Image.new("RGB", (800, 1000), "white") |
| ImageDraw.Draw(page).text((50, 50), "Hello OCR", fill="black") |
| buffer = BytesIO() |
| page.save(buffer, format="PNG") |
| image = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode() |
| session = requests.Session() |
| session.trust_env = False |
| response = session.post("http://127.0.0.1:8000/v1/chat/completions", json={ |
| "model": "./unlimited-ocr-2", |
| "messages": [{"role": "user", "content": [ |
| {"type": "image_url", "image_url": {"url": image}}, |
| {"type": "text", "text": "<PAGE><image>Read this document."}, |
| ]}], |
| "max_tokens": 10, "temperature": 0, |
| }, timeout=120) |
| response.raise_for_status() |
| print(response.json()["choices"][0]["message"]["content"]) |
| ``` |
|
|
| ### Creating this repo |
|
|
| - Code: [.meta/create_model.py](.meta/create_model.py) |
| - Model structure: [.meta/model.log](.meta/model.log) |
| - Environment: [.meta/env.json](.meta/env.json) |
|
|
| ### Change log |
|
|
| - 2026-10-31: Initial tiny Unlimited-OCR 2 checkpoint with the upstream SR-SWA runtime. |
| - 2026-11-01: Separate residual width from the 32-dimensional attention heads to reduce checkpoint size. |
|
|
| *This repo is generated by GPT-6.1-Sol.* |
