| --- |
| base_model: bodhan-ai/indic-ocr |
| library_name: peft |
| pipeline_tag: image-text-to-text |
| tags: |
| - lora |
| - ocr |
| - marathi |
| - gujarati |
| - indicocr |
| language: |
| - mr |
| - gu |
| --- |
| |
| # IndicOCR MarGuj |
|
|
| LoRA for Marathi and Gujarati line OCR on the recognizer inside [bodhan-ai/indic-ocr](https://huggingface.co/bodhan-ai/indic-ocr) (`weights/ocr`, Qwen3.5-0.8B). |
|
|
| The same adapter was trained in two steps. It was first trained on the Marathi real-10k set, then continued for one epoch on 9,200 Gujarati gazette lines. The Gujarati run did not start a new LoRA. Vision weights stay frozen. Base weights are not in this repo. |
|
|
| ## Training |
|
|
| | | Marathi step | Gujarati step | |
| |---|---|---| |
| | Starting weights | `bodhan-ai/indic-ocr` `weights/ocr` | this adapter, after the Marathi step | |
| | Lines | 9,200 real Marathi lines | 9,200 real Gujarati lines | |
| | Optimizer steps | 1,150 | 1,150 | |
| | Batch | 1, gradient accumulation 8 | 1, gradient accumulation 8 | |
| | Learning rate | 5e-5 | 1e-4 | |
| | LoRA | r=16, alpha=32, language-model projections only | same adapter, continued | |
|
|
| Prompt: `Transcribe the text in this image. Write any mathematical expressions in LaTeX, using $...$ for inline math and $$...$$ for display equations.` |
|
|
| Images are area-clamped between 256 and 1536 pixels on a side. Generation uses tokenizer EOS id 262146. |
|
|
| ## Gujarati frozen 100 |
|
|
| Same 100 long lines (`special_chars_expansion.jsonl`) and the same Gujarati metric definitions. PARSeq runs are piecewise. Chitrapathak and IndicOCR runs are whole-line. |
|
|
| | Metric | PARSeq base | PARSeq trained | Chitrapathak base | Chitrapathak MarGuj | IndicOCR base | IndicOCR MarGuj | |
| |---|---:|---:|---:|---:|---:|---:| |
| | CER ↓ | 31.57% | 27.45% | 80.35% | 8.81% | 1.76% | **1.23%** | |
| | Strict WER ↓ | 97.30% | 99.44% | 93.10% | 32.25% | 8.57% | **7.36%** | |
| | Punctuation F1 ↑ | 0.00% | 36.66% | 90.44% | 97.30% | 97.30% | **98.60%** | |
| | Matra ↑ | 84.01% | 91.87% | 0.00% | 92.10% | **98.53%** | **98.53%** | |
| | Conjunct ↑ | 78.70% | 89.51% | 0.00% | 64.20% | 97.22% | **97.53%** | |
| | Numeral ↑ | 54.60% | 82.18% | 0.00% | 92.53% | **96.55%** | 95.98% | |
| | Character accuracy ↑ | 68.43% | 72.55% | 19.65% | 91.19% | 98.24% | **98.77%** | |
| | Exact match ↑ | 0/100 | 0/100 | 0/100 | 5/100 | 43/100 | **46/100** | |
|
|
| ## Gujarati 800-line holdout |
|
|
| Short gazette lines, at most 10 words. This is the split held out from Gujarati training. IndicOCR MarGuj: CER 0.014%, exact match 797/800, vowel signs 100%, conjuncts 100%, numerals 100%, punctuation F1 99.98%. |
|
|
| ## Use |
|
|
| ```python |
| import torch |
| from peft import PeftModel |
| from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration |
| |
| base_id = "bodhan-ai/indic-ocr" |
| adapter_id = "snxtyle/IndicOCR-MarGuj" |
| device = "cuda" if torch.cuda.is_available() else "mps" |
| dtype = torch.bfloat16 |
| |
| model = Qwen3_5ForConditionalGeneration.from_pretrained( |
| base_id, subfolder="weights/ocr", torch_dtype=dtype, attn_implementation="sdpa" |
| ) |
| model = PeftModel.from_pretrained(model, adapter_id).to(device).eval() |
| processor = AutoProcessor.from_pretrained(base_id, subfolder="weights/ocr") |
| |
| prompt = ( |
| "Transcribe the text in this image. Write any mathematical expressions in LaTeX, " |
| "using $...$ for inline math and $$...$$ for display equations." |
| ) |
| messages = [{ |
| "role": "user", |
| "content": [ |
| {"type": "image", "image": image}, |
| {"type": "text", "text": prompt}, |
| ], |
| }] |
| text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = processor(text=[text], images=[image], padding=True, return_tensors="pt").to(device) |
| eos = 262146 |
| with torch.no_grad(): |
| out = model.generate(**inputs, max_new_tokens=512, do_sample=False, eos_token_id=eos) |
| print(processor.batch_decode(out[:, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)[0].strip()) |
| ``` |
|
|
