Aditya-Khedekar/SarvamAI-VLM

---

This model is not merged properly and wont be able to perfom inference on it

This is a finetuned Version of Llama-3.2-11b-VLM model on Indian News paper dataset (https://huggingface.co/datasets/Aditya-Khedekar/SarvamAI-VLM-dataset) With Lora and Unsloth

Baseline vs. Fine-tuned Model

Metric	Baseline	Fine-tuned	Improvement
Average WER	6.0228	0.3940	↓ 93.5%
Average CER	6.9800	0.3442	↓ 95.1%
Exact Sequence Accuracy	0.0047	0.5357	↑ 113x
Flexible Sequence Accuracy	0.0143	0.6137	↑ 42x

The fine-tuning dramatically improved all metrics:

• WER/CER: Error rates decreased by over 90% (lower is better)
• Exact Accuracy: Improved from virtually no exact matches (0.47%) to over 53%
• Flexible Accuracy: Improved from 1.43% to 61.37% of outputs having ≥95% similarity