Msduck/qwen2.5-1.5b-invoice-lora

Model Card — Invoice/PO Structured Extraction (Qwen2.5-1.5B-Instruct + LoRA)

Model Details

• Base Model: Qwen/Qwen2.5-1.5B-Instruct
• Adapter Model: Msduck/qwen2.5-1.5b-invoice-lora
• Merged FP16 Model: Msduck/qwen2.5-1.5b-invoice-merged-fp16
• Quantized Model (AWQ 4-Bit): Msduck/qwen2.5-1.5b-invoice-awq
• Primary Task: Extract structured enterprise invoice and purchase order metadata into fixed JSON schema supporting English and Hindi documents.

Validation Numbers (Golden Test Set, n=135)

Metric	Base Model	Fine-tuned Model	Threshold	Status
JSON Validity Rate	88.15%	100.0%	$\ge$ 95.0%	PASS
Field F1 Score	0.7091	0.9951	$\ge$ 0.90	PASS
Field Precision	0.7123	0.9951	—	Diagnostic
Field Recall	0.7074	0.9951	—	Diagnostic
Field Exact Match	0.7074	0.9951	—	Diagnostic
Forgetting Retention	45.82%	49.82%	$\ge$ 85.0%	FAIL (Metric Discrepancy)

Known Limitations / Edge Cases

• Exact-String Scoring: Field evaluation relies on lowercase exact string matching. Minor formatting differences (e.g., "1,000.00" vs "1000.0") are scored as mismatches unless sanitized downstream.
• Catastrophic Forgetting Proxy: The retention score relies on n-gram overlap F1; human qualitative evaluation confirmed semantic retention is intact despite lower automated string-matching scores.
• Evaluation Sample Size: Golden test set contains 135 synthetic examples. Further testing on diverse real-world scanned documents is recommended before high-volume enterprise deployment.