File size: 3,541 Bytes
809f849 c9cc6a2 809f849 c9cc6a2 9c9c715 c9cc6a2 809f849 c9cc6a2 809f849 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen2.5-3B-Instruct
tags:
- finance
- banking
- rbi
- regulation
- india
- qwen2.5
- lora
datasets:
- Vishva007/RBI-Circular-QA-Dataset
metrics:
- accuracy
---
# Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A
A specialized model for answering questions about **Reserve Bank of India (RBI) regulations and banking policies**.
**Performance**: 57.6% accuracy (8.2x improvement over base model's 7%)
## Quick Facts
- 🎯 **Accuracy**: 57.6% on 1000-sample evaluation set
- 📚 **Coverage**: 100+ regulation areas (Basel III, FEMA, AML, PSL, etc.)
- 🚀 **Training**: 47K QA pairs with rephrased variants
- ⚡ **Efficient**: 3B parameters, optimized for deployment
## Performance Highlights
| Category | Base Model | Fine-tuned | Improvement |
|----------|-----------|-----------|-------------|
| Overall | 7.0% | **57.6%** | +50.6% |
| Fact-based | 6.8% | **57.6%** | +50.8% |
| Reasoning | 37.5% | **62.5%** | +25.0% |
**Top Categories** (70%+ accuracy): Anti-Money Laundering, Digital Payments, Government Banking, MSME Finance, Currency Management
## Training Details
- **Method**: LoRA fine-tuning (r=16, alpha=32)
- **Dataset**: [RBI-Circular-QA-Dataset](https://huggingface.co/datasets/Vishva007/RBI-Circular-QA-Dataset) (47K samples)
- **Training**: 1 epoch, 2 hours on NVIDIA L40S
- **Loss**: 0.79 → 0.57 (train), 0.58 (eval)
## Code & Resources
- 💻 **Training Code**: [GitHub Repository](https://github.com/vishvaRam/Unsloth-FineTuning.git)
- 📊 **Dataset**: [RBI-Circular-QA-Dataset](https://huggingface.co/datasets/Vishva007/RBI-Circular-QA-Dataset)
- 🤗 **Base Model**: [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
## 📝 Technical Deep Dive
Want to understand the theory and methodology behind this model?
**Read the full article:** [Fine-tuning Qwen 2.5 3B for RBI Regulations: Achieving 8x Performance with Smart Data Augmentation](https://dev.to/vishva_ram/fine-tuning-qwen-25-3b-for-rbi-regulations-achieving-8x-performance-with-smart-data-augmentation-5e38)
The article covers:
- 🧠 **LoRA Theory**: Why and how Low-Rank Adaptation works
- ⚡ **Unsloth Deep Dive**: Technical advantages and performance optimizations
- 📊 **Data Augmentation Strategy**: The rephrasing technique that delivered 40% of improvement
- 🎓 **Hyperparameter Analysis**: Detailed explanation of every training choice
- 📈 **Evaluation Methodology**: Stratified sampling and LLM-as-judge approach
- 🔬 **Ablation Studies**: What really mattered for the 8x improvement
Perfect for ML engineers who want to replicate or adapt this approach for their own domain-specific fine-tuning projects.
## Use Cases
✅ Banking compliance chatbots
✅ Regulatory Q&A systems
✅ Training tools for banking professionals
✅ RBI circular analysis
⚠️ Not for: Legal compliance decisions (requires expert review), real-time updates
## Limitations
- Knowledge cutoff: 2024 regulations
- 57.6% accuracy means ~42% of complex queries need verification
- Optimized for English only
## Citation
```
@misc{
qwen25-3b-rbi-qa,
author = {Vishva007},
title = {Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A},
year = {2025},
publisher = {HuggingFace},
howpublished = {\url{https://huggingface.co/Vishva007/Qwen2.5-3B-Instruct-RBI-QA}},
}
```
## License
Apache 2.0 (inherits from base model)
---
**Author**: [@Vishva007](https://huggingface.co/Vishva007) | **Updated**: November 2025
|