---
base_model: datalab-to/chandra-ocr-2
pipeline_tag: image-text-to-text
library_name: llama.cpp
license: other
license_name: ai-pubs-open-rail-m-modified
license_link: https://huggingface.co/datalab-to/chandra-ocr-2/blob/af93b47dba1b47b6640c86ccf487ed2260ab9a09/LICENSE
tags:
  - gguf
  - bf16
  - ocr
  - vision-language
  - llama.cpp
  - rocm
  - amd
  - rdna4
  - gfx1201
---

# Chandra OCR 2 BF16 GGUF — one-GPU RDNA 4 validation

Unofficial, reproducible BF16 GGUF conversion of
[`datalab-to/chandra-ocr-2`](https://huggingface.co/datalab-to/chandra-ocr-2), validated end to end with
[`llama.cpp`](https://github.com/ggml-org/llama.cpp) on one AMD Radeon RX 9070 XT (`gfx1201`).

This is a container conversion, not a fine-tune, new model, or claim of improved OCR quality. The main GGUF preserves
BF16 precision while omitting Chandra's unused multi-token-prediction (MTP) draft head. The vision projector is BF16.
The original model, architecture, training, and authorship belong to Datalab.

## Files

| File | Bytes | SHA-256 |
|---|---:|---|
| `chandra-ocr-2.BF16.gguf` | 9,695,791,648 | `4e9d5fa9854cf820d4425d28034df31ec1221a7f9d1082b0c4359d79f318cb56` |
| `chandra-ocr-2.mmproj-bf16.gguf` | 675,568,864 | `54ddb8285933512cdbf1c84238aa0435b473a6efef2caeda8ca802c2899e87b3` |
| `chat_template.jinja` | 7,622 | `0d158f349ca965f7eea9db0eb45cd177b85bb0e4ae05dcdd0f060da8f7d41812` |

The complete machine-readable provenance is in [`manifest.json`](manifest.json).

## Pinned provenance

- Source model: `datalab-to/chandra-ocr-2`
- Source revision: `af93b47dba1b47b6640c86ccf487ed2260ab9a09`
- Source `model.safetensors` SHA-256: `0804568be9f099d6479fad9ed77a4da4611f3c1e7bc6e009af7dce45e8aa3847`
- Converter/runtime: `ggml-org/llama.cpp`
- Converter revision: `8f5ab832ca7d8a7b4f23687693fb8b0ecbc227e7`
- `chandra-ocr`: `0.2.0`

See [`CONVERSION.md`](CONVERSION.md) for the exact commands and integrity checks.

## Verified configuration

| Component | Verified value |
|---|---|
| GPU | AMD Radeon RX 9070 XT, 16 GB (`gfx1201`) |
| GPU allocation | One isolated GPU; full layer offload |
| OS / kernel | Ubuntu 24.04.4 / 6.17.0-40-generic |
| ROCm | 7.2.1 |
| llama.cpp | `8f5ab832ca7d8a7b4f23687693fb8b0ecbc227e7` |
| Context | 24,576 tokens |
| Maximum tested output envelope | 12,384 tokens |
| Observed model-process VRAM peak | Approximately 11.2 GB |

The GGUF format is not RDNA4-specific. `gfx1201` is the hardware on which this exact pair completed image, PDF,
financial-table, handwritten-document, deterministic-output, and lifecycle tests. Other llama.cpp-supported hardware may
work but is not validated here.

## Run with llama.cpp and the Chandra CLI

Build llama.cpp for your accelerator, then start a loopback server. These are the validated inference settings; replace
the device selector as appropriate for your machine.

```bash
ROCR_VISIBLE_DEVICES=GPU-YOUR-STABLE-UUID HIP_VISIBLE_DEVICES=0 \
llama-server \
  --model chandra-ocr-2.BF16.gguf \
  --mmproj chandra-ocr-2.mmproj-bf16.gguf \
  --alias chandra \
  --host 127.0.0.1 \
  --port 18100 \
  --ctx-size 24576 \
  --n-gpu-layers 999 \
  --split-mode none \
  --main-gpu 0 \
  --flash-attn on \
  --fit off \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --image-min-tokens 1024
```

In another shell:

```bash
python3 -m venv .venv
.venv/bin/pip install 'chandra-ocr==0.2.0'

VLLM_API_BASE=http://127.0.0.1:18100/v1 \
VLLM_API_KEY=EMPTY \
VLLM_MODEL_NAME=chandra \
.venv/bin/python -m chandra.scripts.cli input.pdf output \
  --method vllm \
  --batch-size 1 \
  --max-workers 1 \
  --max-retries 2 \
  --max-output-tokens 12384 \
  --no-images \
  --no-html
```

Do not expose an unauthenticated llama.cpp server to a public network. Generative OCR can omit or hallucinate content;
verify consequential documents against their source.

## Validation result and limits

The exact-pinned conversion matched the public comparison BF16 tensor schema. All projector tensor payloads matched. In
the main file, every tensor payload matched except twelve scalar F32 values across ten `ssm_a` tensors; the maximum
absolute difference was `9.5367431640625e-07`. Accepted OCR outputs matched. There is no evidence that this conversion
improves OCR quality over other correct BF16 conversions.

The useful contribution is the pinned, MTP-free artifact pair; complete provenance; one-16-GB-GPU validation; and a
reproducible launch envelope.

## License and attribution

Chandra OCR 2 weights use Datalab's **AI Pubs Open RAIL-M License (Modified)**. It contains usage, redistribution,
commercial, competitive-use, attribution, and share-alike conditions. Read [`LICENSE`](LICENSE) in full before using or
redistributing these files. The license and its restrictions apply to this derivative conversion.

Modified-file notice: the original Chandra OCR 2 checkpoint was converted to GGUF at the pinned llama.cpp revision; the
unused MTP draft head was omitted from the main GGUF; model tensor precision otherwise remains BF16. No Datalab
endorsement is claimed.