Instructions to use taresco/KarantaOCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use taresco/KarantaOCR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="taresco/KarantaOCR")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("taresco/KarantaOCR")
model = AutoModelForMultimodalLM.from_pretrained("taresco/KarantaOCR", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use taresco/KarantaOCR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "taresco/KarantaOCR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "taresco/KarantaOCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/taresco/KarantaOCR

SGLang

How to use taresco/KarantaOCR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "taresco/KarantaOCR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "taresco/KarantaOCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "taresco/KarantaOCR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "taresco/KarantaOCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use taresco/KarantaOCR with Docker Model Runner:
```
docker model run hf.co/taresco/KarantaOCR
```

ToluClassics commited on Dec 25, 2025

Commit

4cfa60d

verified ·

1 Parent(s): 429e471

Update README.md

Browse files

Files changed (1) hide show

README.md +26 -0

README.md CHANGED Viewed

@@ -202,3 +202,29 @@ def run_inference(model, processor, messages, max_new_tokens=128, device="cuda")
     )
     return outputs[0]
 ```

     )
     return outputs[0]
 ```
+### End-to-End Example
+```python
+model = load_model("taresco/KarantaOCR")
+processor = load_processor("taresco/KarantaOCR")
+prompt = """Below is the image of one page of a PDF document.
+Just return the plain text representation of this document as if you were reading it naturally.
+Turn equations into a LaTeX representation, and tables into markdown format. Remove the headers and footers, but keep references and footnotes.
+Read any natural handwriting.
+This is likely one page out of several in the document, so be sure to preserve any sentences that come from the previous page, or continue onto the next page, exactly as they are.
+If there is no text at all that you think you should read, you can output null.
+if the document contains diacritics, please include them in the output.
+Do not hallucinate.
+"""
+messages = build_message(
+    image_url="example.pdf",
+    system_prompt=prompt,
+    page=0
+)
+output_text = run_inference(model, processor, messages)
+print(output_text)
+```