Create README.md

README.md

@@ -0,0 +1,215 @@
+# AssameseOCR
+
+**AssameseOCR** is a vision-language model for Optical Character Recognition (OCR) of printed Assamese text. Built on Microsoft's Florence-2-large foundation model with a custom character-level decoder, it achieves 94.67% character accuracy on the Mozhi dataset.
+
+## Model Details
+
+### Model Description
+
+- **Developed by:** MWire Labs
+- **Model type:** Vision-Language OCR
+- **Language:** Assamese (অসমীয়া)
+- **License:** Apache 2.0
+- **Base Model:** microsoft/Florence-2-large-ft
+- **Architecture:** Florence-2 Vision Encoder + Custom Transformer Decoder
+
+### Model Architecture
+
+```
+Image (768×768) 
+  ↓
+Florence-2 Vision Encoder (frozen, 360M params)
+  ↓
+Vision Projection (1024 → 512 dim)
+  ↓
+Transformer Decoder (4 layers, 8 heads)
+  ↓
+Character-level predictions (187 vocab)
+```
+
+**Key Components:**
+- **Vision Encoder:** Florence-2-large DaViT architecture (frozen)
+- **Decoder:** 4-layer Transformer with 512 hidden dimensions
+- **Tokenizer:** Character-level with 187 tokens (Assamese chars + English + digits + symbols)
+- **Total Parameters:** 378M (361M frozen, 17.5M trainable)
+
+## Training Details
+
+### Training Data
+
+- **Dataset:** [Mozhi Indic OCR Dataset](https://huggingface.co/datasets/darknight054/indic-mozhi-ocr) (Assamese subset)
+- **Training samples:** 79,697 word images
+- **Validation samples:** 9,945 word images
+- **Test samples:** 10,146 word images
+- **Source:** IIT Hyderabad CVIT
+
+### Training Procedure
+
+**Hardware:**
+- GPU: NVIDIA A40 (48GB VRAM)
+- Training time: ~8 hours (3 epochs)
+
+**Hyperparameters:**
+- Epochs: 3
+- Batch size: 16
+- Learning rate: 3e-4
+- Optimizer: AdamW (weight_decay=0.01)
+- Scheduler: CosineAnnealingLR
+- Max sequence length: 128 characters
+- Gradient clipping: 1.0
+
+**Training Strategy:**
+- Froze Florence-2 vision encoder (leveraging pretrained visual features)
+- Trained only the projection layer and transformer decoder
+- Full fine-tuning (no LoRA) for maximum quality
+
+## Performance
+
+### Results
+
+| Split | Character Accuracy | Loss |
+|-------|-------------------|------|
+| Epoch 1 (Val) | 91.61% | 0.2844 |
+| Epoch 2 (Val) | 94.09% | 0.1548 |
+| Epoch 3 (Val) | **94.67%** | **0.1221** |
+
+**Character Error Rate (CER):** ~5.33%
+
+### Comparison
+
+The model achieves strong performance for a foundation model approach:
+- Mozhi paper (CRNN+CTC specialist): ~99% accuracy
+- AssameseOCR (Florence generalist): 94.67% accuracy
+
+The 5% gap is expected when adapting a general vision-language model versus training a specialized OCR architecture. However, AssameseOCR offers:
+- Extensibility to vision-language tasks (VQA, captioning, document understanding)
+- Faster training (3 epochs vs typical 10-20 for CRNN)
+- Foundation model benefits (transfer learning, robustness)
+
+## Usage
+
+### Installation
+
+```bash
+pip install torch torchvision transformers pillow
+```
+
+### Inference
+
+```python
+import torch
+from PIL import Image
+from transformers import AutoModelForCausalLM, CLIPImageProcessor
+import json
+
+# Load tokenizer
+class CharTokenizer:
+    def __init__(self, vocab):
+        self.vocab = vocab
+        self.char2id = {c: i for i, c in enumerate(vocab)}
+        self.id2char = {i: c for i, c in enumerate(vocab)}
+        self.pad_token_id = self.char2id["<pad>"]
+        self.bos_token_id = self.char2id["<s>"]
+        self.eos_token_id = self.char2id["</s>"]
+        
+    def decode(self, ids, skip_special_tokens=True):
+        chars = []
+        for i in ids:
+            ch = self.id2char.get(i, "")
+            if skip_special_tokens and ch.startswith("<"):
+                continue
+            chars.append(ch)
+        return "".join(chars)
+    
+    @classmethod
+    def load(cls, path):
+        with open(path, "r", encoding="utf-8") as f:
+            vocab = json.load(f)
+        return cls(vocab)
+
+# Load model components
+device = "cuda" if torch.cuda.is_available() else "cpu"
+
+# Load Florence base model
+florence_model = AutoModelForCausalLM.from_pretrained(
+    "microsoft/Florence-2-large-ft",
+    trust_remote_code=True
+).to(device)
+
+# Load image processor
+image_processor = CLIPImageProcessor.from_pretrained("microsoft/Florence-2-large-ft")
+
+# Load tokenizer
+char_tokenizer = CharTokenizer.load("assamese_char_tokenizer.json")
+
+# Load AssameseOCR weights
+# (Note: You'll need to define the FlorenceCharOCR class as in training)
+checkpoint = torch.load("assamese_ocr_best.pt", map_location=device)
+# ocr_model.load_state_dict(checkpoint['model_state_dict'])
+
+# Inference
+image = Image.open("assamese_text.jpg")
+# Process and predict...
+```
+
+## Vocabulary
+
+The character-level tokenizer includes:
+- **Assamese characters:** 119 unique chars (consonants, vowels, diacritics, conjuncts)
+- **English:** 52 chars (a-z, A-Z)
+- **Digits:** 30 chars (ASCII 0-9, Assamese ০-৯, Devanagari ०-९)
+- **Symbols:** 33 chars (punctuation, special chars)
+- **Special tokens:** 6 tokens (`<pad>`, `<s>`, `</s>`, `<unk>`, `<OCR>`, `<lang_as>`)
+- **Total vocabulary:** 187 tokens
+
+## Limitations
+
+- Trained only on printed text (not handwritten)
+- Word-level images from Mozhi dataset (may not generalize to full-page OCR without line segmentation)
+- Character-level decoder may struggle with very long sequences (>128 chars)
+- Does not handle layout analysis or reading order
+- Performance on degraded/low-quality images not extensively tested
+
+## Future Work
+
+- Extend to **MeiteiOCR** for Meitei Mayek script
+- Scale to **NE-OCR** covering all 9+ Northeast Indian languages
+- Add document layout analysis and reading order detection
+- Improve performance with synthetic data augmentation
+- Fine-tune for handwritten text recognition
+- Extend to multimodal tasks (image captioning, VQA for documents)
+
+## Citation
+
+If you use AssameseOCR in your research, please cite:
+
+```bibtex
+@software{assameseocr2026,
+  author = {MWire Labs},
+  title = {AssameseOCR: Vision-Language Model for Assamese Text Recognition},
+  year = {2026},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/MWirelabs/assamese-ocr}
+}
+```
+
+## Acknowledgments
+
+- **Dataset:** Mozhi Indic OCR Dataset by IIT Hyderabad CVIT ([Mathew et al., 2022](https://arxiv.org/abs/2205.06740))
+- **Base Model:** Florence-2 by Microsoft Research
+- **Organization:** MWire Labs, Shillong, Meghalaya, India
+
+## Contact
+
+- **Organization:** [MWire Labs](https://huggingface.co/MWirelabs)
+- **Location:** Shillong, Meghalaya, India
+- **Focus:** Language technology for Northeast Indian languages
+
+---
+
+**Part of the MWire Labs NLP suite:**
+- [KhasiBERT](https://huggingface.co/MWirelabs/KhasiBERT-110M) - Khasi language model
+- [NE-BERT](https://huggingface.co/MWirelabs/NE-BERT) - 9 Northeast languages
+- [Kren-M](https://huggingface.co/MWirelabs/Kren-M) - Khasi-English conversational AI
+
+- **AssameseOCR** - Assamese text recognition