--- library_name: transformers base_model: - datalab-to/chandra-ocr-2 license: openrail license_link: https://huggingface.co/datalab-to/chandra-ocr-2/blob/main/LICENSE tags: - llama-cpp - mtp - ocr - pdf - markdown - layout - text-generation-inference - llama-cli - repack - HTML - JSON language: - en pipeline_tag: image-text-to-text --- # **chandra-ocr-2-MTP-Repacked** > Chandra-OCR-2 from Datalab is a state-of-the-art OCR model that outputs structured markdown, HTML, or JSON while preserving precise layout information from images and PDFs across 90+ languages. It achieves SOTA benchmarks with 85.9% on olmocr and 77.8% multilingual score (+12% over Chandra 1), delivering major gains in math equation parsing, complex table reconstruction (including merged cells), handwriting recognition, form elements like checkboxes, and wide-document layouts alongside vastly improved image captioning and diagram extraction. Available via free playground, hosted API for production speed/accuracy, or local deployment through HuggingFace Transformers/vLLM, it excels at transforming challenging real-world documents—financial filings, research papers, historical scans, multilingual forms—into semantically rich structured data for downstream AI pipelines and automation workflows. > [!NOTE] > MTP — Speculative Decoding: Yes | GGUF — [chandra-ocr-2-MTP-Repacked(GGUF)](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/tree/main/GGUF) > [!NOTE] Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass. ## Model Files | File Name | Quant Type | File Size | File Link | Description | |-----------|------------|-----------|-----------|-------------| | chandra-ocr-2-MTP.BF16.gguf | BF16 | 8.67 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.BF16.gguf) | Full BF16 weights. Highest quality, largest file size. | | chandra-ocr-2-MTP.Q3_K_L.gguf | Q3_K_L | 2.48 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q3_K_L.gguf) | Lower quality but usable, good for low RAM availability. | | chandra-ocr-2-MTP.Q3_K_M.gguf | Q3_K_M | 2.32 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q3_K_M.gguf) | Low quality. | | chandra-ocr-2-MTP.Q4_K_M.gguf | Q4_K_M | 2.78 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q4_K_M.gguf) | Good quality, default size for most use cases, *recommended*. | | chandra-ocr-2-MTP.Q4_K_S.gguf | Q4_K_S | 2.63 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q4_K_S.gguf) | Slightly lower quality with more space savings, *recommended*. | | chandra-ocr-2-MTP.Q5_K_M.gguf | Q5_K_M | 3.16 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q5_K_M.gguf) | High quality, *recommended*. | | chandra-ocr-2-MTP.Q5_K_S.gguf | Q5_K_S | 3.07 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q5_K_S.gguf) | High quality, *recommended*. | | chandra-ocr-2-MTP.Q6_K.gguf | Q6_K | 3.56 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q6_K.gguf) | Very high quality, near perfect, *recommended*. | | chandra-ocr-2-MTP.Q8_0.gguf | Q8_0 | 4.61 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.Q8_0.gguf) | Extremely high quality, generally unneeded but max available quant. | | chandra-ocr-2-MTP.mmproj-bf16.gguf | mmproj-bf16 | 676 MB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/GGUF/chandra-ocr-2-MTP.mmproj-bf16.gguf) | Multimodal projection file in BF16 format. Used for vision/OCR capabilities. | ## MTP-Repacked [Transformers, BF16] | File Name | File Size | File Link | Description | |-----------|-----------|-----------|-------------| | chat_template.jinja | 7.62 kB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/chat_template.jinja) | Jinja template for chat formatting. | | config.json | 2.88 kB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/config.json) | Model configuration file. | | generation_config.json | 116 Bytes | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/generation_config.json) | Generation configuration parameters. | | model-00001-of-00002.safetensors | 5.07 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/model-00001-of-00002.safetensors) | Model weights shard 1 of 2 (Safetensors format). | | model-00002-of-00002.safetensors | 4.25 GB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/model-00002-of-00002.safetensors) | Model weights shard 2 of 2 (Safetensors format). | | model.safetensors.index.json | 67.3 kB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/model.safetensors.index.json) | Index file mapping weights to shards. | | processor_config.json | 1.33 kB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/processor_config.json) | Processor configuration for multimodal inputs. | | tokenizer.json | 20 MB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/tokenizer.json) | Tokenizer vocabulary and merges. | | tokenizer_config.json | 1.51 kB | [Link](https://huggingface.co/prithivMLmods/chandra-ocr-2-MTP-Repacked/blob/main/tokenizer_config.json) | Tokenizer configuration file. | ## BF16 Test Run Tested `chandra-ocr-2-MTP.BF16.gguf` (with `chandra-ocr-2-MTP.mmproj-bf16.gguf`) on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with `96 GB` of VRAM, using `llama-cli` with full GPU offloading and MTP speculative decoding. ### Performance Metrics | Metric | Value | | --- | --- | | Average Prompt Processing Speed | 2,383.63 tokens/s | | Average Generation Speed | 196.98 tokens/s | | Total Wall Time | 25.4 seconds | | Context Length | 16,384 tokens | | KV Cache (K) | `q8_0` | | GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition | | GPU VRAM Capacity | 96 GB | | Peak GPU VRAM Usage | 10.66 GiB | | Peak GPU Utilization | 87% | | GPU Layers Offloaded | 99 (`-ngl 99`) | | Random Seed | 42 | | Speculative Decoding | MTP (`draft_n_max=2`) | | Test Runs | 3 | ### Test Shots | Shot | Test | | --- | --- | | Shot 1 | Vision OCR — Image 1 | | Shot 2 | Vision OCR — Image 2 | | Shot 3 | Vision OCR — Image 3 | ### Run Inference ```bash ./build/bin/llama-cli \ -m /path/to/models/chandra-ocr-2-MTP.BF16.gguf \ --mmproj /path/to/models/chandra-ocr-2-MTP.mmproj-bf16.gguf \ --image /path/to/image.jpeg \ --spec-type draft-mtp \ --spec-draft-n-max 2 \ -ngl 99 \ -c 16384 \ --cache-type-k q8_0 \ --seed 42 \ -n 8192 \ -st \ -p "OCR the image." ``` *Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.* ## Snapshot(3) ![download (1)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/WW1AUsd2y-nw1cpEr6W8P.png) ![download (2)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/-1lubFDLfx3-XuM_6YZka.png) ![download (3)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/StqYuQ406HJH67k2_YrmE.png) ![download](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/EaWYBEfU3uzXaoh4MCdCp.png) ## llama.cpp LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp MTP-repackaged and quantized using llama.cpp by — [prithivMLmods](https://huggingface.co/prithivMLmods).