| --- |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| base_model: |
| - openbmb/MiniCPM-OCR-Jev |
| --- |
| |
| This is a randomly initialized tiny model based on [openbmb/MiniCPM-OCR-Jev](https://huggingface.co/openbmb/MiniCPM-OCR-Jev) for testing and debugging. |
|
|
| | Feature | Source | This repo | |
| |---|---|---| |
| | Checkpoint tensor elements | 2.21B BF16 | 6.64M BF16 | |
| | Safetensors weight files | 4.43 GB (model.safetensors) | 13.31 MB (model.safetensors) | |
| | Architecture / input | Qwen3.5 vision-language correctness scorer | Same | |
| | Vocabulary rows / tied embeddings | 248,320 / false | Same | |
| | Backbone layers / layer cadence | 24 / six 3-linear + 1-full cycles | 8 / two complete cycles | |
| | Hidden width / MLP width | 2,048 / 6,144 | 16 / 32 | |
| | Q heads / KV heads / head dimension | 8 / 2 / 256 | Same | |
| | Linear-attention key/value heads and dimensions | 16/16, 128/128 | Same | |
| | Vision depth / width / output width | 24 / 1,024 / 2,048 | 4 / 128 / 16 | |
| | Vision patch / temporal patch / spatial merge | 16 / 2 / 2 | Same | |
| | Task head | binary `score` [2, 2,048]; no `lm_head` | binary `score` [2, 16]; no `lm_head` | |
| | MTP | config declares one layer; no MTP tensors | Same | |
| | Checkpoint precision | BF16 | Same | |
|
|
| Load this checkpoint with `AutoModelForSequenceClassification`; it keeps the source binary `score` head and does not contain `lm_head`. Two complete linear/linear/linear/full cycles are retained. The Qwen3.5 linear-attention path requires CUDA and Triton. The exact public snippet was verified on an NVIDIA A800-SXM4-80GB. |
|
|
| | File path | Size | |
| |------|------| |
| | model.safetensors | 13.3MB | |
|
|
| ### Example usage |
|
|
| #### Transformers (✅ verified) |
|
|
| The example sends two images: the original block and a rendered image of the candidate text. Load with `AutoModelForSequenceClassification`; the checkpoint has a binary `score` head instead of `lm_head`. Install `transformers>=5.10.4` and `Pillow`; the Qwen3.5 linear-attention path requires CUDA and Triton. |
|
|
| ```python |
| def run_transformers(model_id: str): |
| import torch |
| from PIL import Image, ImageDraw |
| from transformers import AutoModelForSequenceClassification, AutoProcessor |
| |
| model_id = str(model_id) |
| processor = AutoProcessor.from_pretrained(model_id) |
| model = AutoModelForSequenceClassification.from_pretrained( |
| model_id, |
| dtype=torch.bfloat16, |
| device_map="cuda", |
| attn_implementation="sdpa", |
| ) |
| candidate = "Hello" |
| original = Image.new("RGB", (448, 448), "white") |
| ImageDraw.Draw(original).text((20, 20), candidate, fill="black") |
| rendered = Image.new("RGB", (448, 448), "white") |
| ImageDraw.Draw(rendered).text((20, 20), candidate, fill="black") |
| instruction = ( |
| "第一张图是原图，第二张图是上述 OCR 文本渲染得到的图片。\n" |
| "请判断该 OCR 文本及其对应渲染图是否与原图内容一致。\n" |
| "这是一个二分类任务：一致标为 1，不一致标为 0。" |
| ) |
| messages = [ |
| { |
| "role": "user", |
| "content": [ |
| {"type": "image", "image": original}, |
| {"type": "text", "text": f"\nOCR文本：\n{candidate}\n"}, |
| {"type": "image", "image": rendered}, |
| {"type": "text", "text": "\n" + instruction}, |
| ], |
| } |
| ] |
| inputs = processor.apply_chat_template( |
| messages, |
| tokenize=True, |
| add_generation_prompt=True, |
| return_dict=True, |
| return_tensors="pt", |
| ).to(model.device) |
| with torch.no_grad(): |
| logits = model(**inputs).logits |
| assert torch.isfinite(logits).all() |
| score = torch.softmax(logits.float(), dim=-1)[:, 1] |
| print(float(score[0])) |
| |
| |
| if __name__ == "__main__": |
| run_transformers(model_id="tiny-random/minicpm-ocr-jev") |
| ``` |
|
|
| ### Creating this repo |
|
|
| - Code: [.meta/create_model.py](.meta/create_model.py) |
| - Model structure: [.meta/model.log](.meta/model.log) |
| - Environment: [.meta/env.json](.meta/env.json) |
|
|
| ### Change log |
|
|
| - 2026-11-01: Initial tiny MiniCPM-OCR Jev checkpoint. |
|
|
| *This repo is generated by DeepSeek-V4.1-Flash.* |
