File size: 5,427 Bytes
facb9f1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | ---
library_name: transformers
pipeline_tag: image-text-to-text
base_model:
- baidu/Unlimited-OCR-2
---
This is a randomly initialized tiny model based on [baidu/Unlimited-OCR-2](https://huggingface.co/baidu/Unlimited-OCR-2) for testing and debugging.
| Feature | Source | This repo |
|---|---|---|
| Checkpoint tensor elements | 3.34B BF16, 257 I64 | 9.04M BF16, 257 I64 |
| Safetensors weight files | 6.69 GB (2 files) | 18.12 MB (model.safetensors) |
| Decoder layers / hidden width / dense MLP | 12 / 1,280 / 6,848 | 4 / 32 / 256 |
| Q / KV heads / head dimension | 10 / 10 / 128 | 4 / 4 / 32 |
| Routed experts / top-k / shared experts | 64 / 6 / 2 | 16 / 6 / 2 |
| Expert intermediate width / initial dense layers | 896 / 1 | 64 / 1 |
| DASH index width / sparse block size | 128 / 32 | 32 / 32 |
| Page and token RoPE / sparse layer coverage | page2d / all layers | Same |
| SAM layers / width / global attention layers | 12 / 768 / 2,5,8,11 | 6 / 32 / 2,5 |
| CLIP layers / width | 24 / 1,024 | 4 / 32 |
| SAM neck / downsampling channels | 256 / 512,1,024 | 8 / 16,32 |
| Vision projector input / output | 2,048 / 1,280 | 64 / 32 |
| Global / crop image size / page visual tokens | 1,024 / 640 / 273 | Same |
| Vocabulary rows / tied embeddings | 129,280 / no | Same |
| Serialized precision | BF16 weights + I64 position IDs | Same |
Residual width 32 is independent of the four 32-dimensional attention heads: Q/K/V project to 128 channels and the output projection returns to the residual width. The included runtime exposes these config dimensions while preserving the upstream attention kernels.
| File path | Size |
|------|------|
| model.safetensors | 18.1MB |
### Example usage
#### Transformers (✅ verified)
Install Transformers 4.52.1, `torchvision`, `Pillow`, `addict`, `easydict` and `einops` and load the included upstream remote code. The source registers only `mha_eager`; its OCR API runs DASH Triton sparse prefill, the reference masked sparse decode, and SDPA in SAM/CLIP.
```python
def run_transformers(model_id: str):
import torch
from PIL import Image, ImageDraw
from transformers import AutoModel, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True, torch_dtype=torch.bfloat16).eval().cuda()
page = Image.new("RGB", (800, 1000), "white")
ImageDraw.Draw(page).text((50, 50), "Hello OCR. Page one.", fill="black")
text = model.infer(tokenizer, prompt="Read this document.", image_file=page, max_new_tokens=10)
print(text)
text, tokens = model.infer_multi(tokenizer, image_files=[page, page.copy()], max_new_tokens=10)
print(text, tokens)
if __name__ == "__main__":
run_transformers(model_id="tiny-random/unlimited-ocr-2")
```
#### SGLang (✅ verified)
Install Baidu's pinned `sglang-0.5.20.dev13+g75b9b06b02` wheel from the [source repo](https://huggingface.co/baidu/Unlimited-OCR-2/tree/c14071b549f4fcab10909f3c4acc6a2edb366947/wheel). The included external-model package reads the reduced vision config, separates residual width from attention head dimension, and gathers the TP-sharded DASH index projections before normalization. The upstream attention and routing kernels are retained.
```bash
hf download tiny-random/unlimited-ocr-2 --local-dir ./unlimited-ocr-2
pip install --no-deps ./unlimited-ocr-2
export SGLANG_EXTERNAL_MODEL_PACKAGE=unlimited_ocr_runtime
export SGLANG_UOCR_EANABLE_DOC_LIMIT=0
python -m sglang.launch_server \
--model-path ./unlimited-ocr-2 \
--tp-size 4 \
--trust-remote-code \
--chat-template unlimited-ocr \
--context-length 4096 \
--max-total-tokens 8192 \
--max-running-requests 4 \
--cuda-graph-backend-decode full \
--cuda-graph-max-bs-decode 4 \
--cuda-graph-backend-prefill full \
--cuda-graph-max-bs-prefill 2048 \
--cuda-graph-bs-prefill 2048 \
--max-prefill-tokens 2048 \
--chunked-prefill-size 2048 \
--cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 4}}' \
--mem-fraction-static 0.3 \
--host 0.0.0.0 --port 8000
```
```python
import base64
from io import BytesIO
import requests
from PIL import Image, ImageDraw
page = Image.new("RGB", (800, 1000), "white")
ImageDraw.Draw(page).text((50, 50), "Hello OCR", fill="black")
buffer = BytesIO()
page.save(buffer, format="PNG")
image = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
session = requests.Session()
session.trust_env = False
response = session.post("http://127.0.0.1:8000/v1/chat/completions", json={
"model": "./unlimited-ocr-2",
"messages": [{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": image}},
{"type": "text", "text": "<PAGE><image>Read this document."},
]}],
"max_tokens": 10, "temperature": 0,
}, timeout=120)
response.raise_for_status()
print(response.json()["choices"][0]["message"]["content"])
```
### Creating this repo
- Code: [.meta/create_model.py](.meta/create_model.py)
- Model structure: [.meta/model.log](.meta/model.log)
- Environment: [.meta/env.json](.meta/env.json)
### Change log
- 2026-10-31: Initial tiny Unlimited-OCR 2 checkpoint with the upstream SR-SWA runtime.
- 2026-11-01: Separate residual width from the 32-dimensional attention heads to reduce checkpoint size.
*This repo is generated by GPT-6.1-Sol.* |