Nzyoka19/nyansapo-ocr-trocr-lora_v1

Nyansapo OCR — TrOCR LoRA v1

Fine-tuned OCR model by Nyansapo built on microsoft/trocr-base-handwritten with LoRA adapters on the decoder.

Training details

• Base model: microsoft/trocr-base-handwritten
• PEFT: LoRA (r=16, alpha=32)
• Targets: q_proj, k_proj, v_proj, out_proj, fc1, fc2
• Dataset: Nzyoka19/ocr_images

Inference

from transformers import VisionEncoderDecoderModel, TrOCRProcessor
from peft import PeftModel
from PIL import Image

base         = VisionEncoderDecoderModel.from_pretrained('microsoft/trocr-base-handwritten')
processor    = TrOCRProcessor.from_pretrained('Nzyoka19/nyansapo-ocr-trocr-lora_v1')
base.decoder = PeftModel.from_pretrained(base.decoder, 'Nzyoka19/nyansapo-ocr-trocr-lora_v1')
base.eval()

image         = Image.open('image.png').convert('RGB')
pixel_values  = processor(images=image, return_tensors='pt').pixel_values
ids           = base.generate(pixel_values)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])