Instructions to use lightonai/LightOnOCR-2-1B-bbox with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lightonai/LightOnOCR-2-1B-bbox with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="lightonai/LightOnOCR-2-1B-bbox")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForSeq2SeqLM

processor = AutoProcessor.from_pretrained("lightonai/LightOnOCR-2-1B-bbox")
model = AutoModelForSeq2SeqLM.from_pretrained("lightonai/LightOnOCR-2-1B-bbox")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lightonai/LightOnOCR-2-1B-bbox with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lightonai/LightOnOCR-2-1B-bbox"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightonai/LightOnOCR-2-1B-bbox",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lightonai/LightOnOCR-2-1B-bbox

SGLang

How to use lightonai/LightOnOCR-2-1B-bbox with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lightonai/LightOnOCR-2-1B-bbox" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightonai/LightOnOCR-2-1B-bbox",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lightonai/LightOnOCR-2-1B-bbox" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightonai/LightOnOCR-2-1B-bbox",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use lightonai/LightOnOCR-2-1B-bbox with Docker Model Runner:
```
docker model run hf.co/lightonai/LightOnOCR-2-1B-bbox
```

Multi-stage OCR of large images

by Albie94 - opened Jan 24

Discussion

Albie94

Jan 24

Can I use this model for multi-stage OCR of large images:

Stage - Layout and Reading order bbox detection like Surya OCR
Stage - Cut image areas by bbox and OCR it by reading order
Stage - Format and concatination final OCR result

staghado

LightOn AI org Jan 27

Hello,
This model is meant for full-page OCR without needing an external pipeline, but is compatible with one, you can just feed the crops to the model.

Albie94

Jan 31

•

edited Jan 31

Hello,
This model is meant for full-page OCR without needing an external pipeline, but is compatible with one, you can just feed the crops to the model.

I couldn't find a way to get only bboxes in LightOnOCR models. Without this, it is impossible to correctly OCR large images (some pdf magazines ), using only LightOnOCR model in llama.cpp gguf on notebook (CPU/GPU). For example, with "prompt_layout_only_en" from dots.ocr I can reduce the image to 4500px, get the bboxes, then recalculate them in the original image, cut each bbox, reduce it to 4500px and OCR it. If LightOnOCR had a similar feature (special prompt), it would probably be the only model in the world that works on a regular laptop with such high quality. It’s interesting that at the moment LightOnOCR-2-1B, according to my measurements, shows much lower OCR quality on such popular large-sized magazines than LightOnOCR-1B-1025.

Albie94 changed discussion status to closed Feb 11

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment