|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: apache-2.0 |
|
|
base_model: Qwen/Qwen2.5-3B-Instruct |
|
|
tags: |
|
|
- finance |
|
|
- banking |
|
|
- rbi |
|
|
- regulation |
|
|
- india |
|
|
- qwen2.5 |
|
|
- lora |
|
|
datasets: |
|
|
- Vishva007/RBI-Circular-QA-Dataset |
|
|
metrics: |
|
|
- accuracy |
|
|
--- |
|
|
|
|
|
# Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A |
|
|
|
|
|
A specialized model for answering questions about **Reserve Bank of India (RBI) regulations and banking policies**. |
|
|
|
|
|
**Performance**: 57.6% accuracy (8.2x improvement over base model's 7%) |
|
|
|
|
|
## Quick Facts |
|
|
|
|
|
- 🎯 **Accuracy**: 57.6% on 1000-sample evaluation set |
|
|
- 📚 **Coverage**: 100+ regulation areas (Basel III, FEMA, AML, PSL, etc.) |
|
|
- 🚀 **Training**: 47K QA pairs with rephrased variants |
|
|
- ⚡ **Efficient**: 3B parameters, optimized for deployment |
|
|
|
|
|
|
|
|
## Performance Highlights |
|
|
|
|
|
| Category | Base Model | Fine-tuned | Improvement | |
|
|
|----------|-----------|-----------|-------------| |
|
|
| Overall | 7.0% | **57.6%** | +50.6% | |
|
|
| Fact-based | 6.8% | **57.6%** | +50.8% | |
|
|
| Reasoning | 37.5% | **62.5%** | +25.0% | |
|
|
|
|
|
**Top Categories** (70%+ accuracy): Anti-Money Laundering, Digital Payments, Government Banking, MSME Finance, Currency Management |
|
|
|
|
|
## Training Details |
|
|
|
|
|
- **Method**: LoRA fine-tuning (r=16, alpha=32) |
|
|
- **Dataset**: [RBI-Circular-QA-Dataset](https://huggingface.co/datasets/Vishva007/RBI-Circular-QA-Dataset) (47K samples) |
|
|
- **Training**: 1 epoch, 2 hours on NVIDIA L40S |
|
|
- **Loss**: 0.79 → 0.57 (train), 0.58 (eval) |
|
|
|
|
|
## Code & Resources |
|
|
|
|
|
- 💻 **Training Code**: [GitHub Repository](https://github.com/vishvaRam/Unsloth-FineTuning.git) |
|
|
- 📊 **Dataset**: [RBI-Circular-QA-Dataset](https://huggingface.co/datasets/Vishva007/RBI-Circular-QA-Dataset) |
|
|
- 🤗 **Base Model**: [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) |
|
|
|
|
|
## 📝 Technical Deep Dive |
|
|
|
|
|
Want to understand the theory and methodology behind this model? |
|
|
|
|
|
**Read the full article:** [Fine-tuning Qwen 2.5 3B for RBI Regulations: Achieving 8x Performance with Smart Data Augmentation](https://dev.to/vishva_ram/fine-tuning-qwen-25-3b-for-rbi-regulations-achieving-8x-performance-with-smart-data-augmentation-5e38) |
|
|
|
|
|
The article covers: |
|
|
- 🧠 **LoRA Theory**: Why and how Low-Rank Adaptation works |
|
|
- ⚡ **Unsloth Deep Dive**: Technical advantages and performance optimizations |
|
|
- 📊 **Data Augmentation Strategy**: The rephrasing technique that delivered 40% of improvement |
|
|
- 🎓 **Hyperparameter Analysis**: Detailed explanation of every training choice |
|
|
- 📈 **Evaluation Methodology**: Stratified sampling and LLM-as-judge approach |
|
|
- 🔬 **Ablation Studies**: What really mattered for the 8x improvement |
|
|
|
|
|
Perfect for ML engineers who want to replicate or adapt this approach for their own domain-specific fine-tuning projects. |
|
|
|
|
|
## Use Cases |
|
|
|
|
|
✅ Banking compliance chatbots |
|
|
✅ Regulatory Q&A systems |
|
|
✅ Training tools for banking professionals |
|
|
✅ RBI circular analysis |
|
|
|
|
|
⚠️ Not for: Legal compliance decisions (requires expert review), real-time updates |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- Knowledge cutoff: 2024 regulations |
|
|
- 57.6% accuracy means ~42% of complex queries need verification |
|
|
- Optimized for English only |
|
|
|
|
|
## Citation |
|
|
|
|
|
``` |
|
|
@misc{ |
|
|
qwen25-3b-rbi-qa, |
|
|
author = {Vishva007}, |
|
|
title = {Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A}, |
|
|
year = {2025}, |
|
|
publisher = {HuggingFace}, |
|
|
howpublished = {\url{https://huggingface.co/Vishva007/Qwen2.5-3B-Instruct-RBI-QA}}, |
|
|
} |
|
|
``` |
|
|
|
|
|
|
|
|
## License |
|
|
|
|
|
Apache 2.0 (inherits from base model) |
|
|
|
|
|
--- |
|
|
|
|
|
**Author**: [@Vishva007](https://huggingface.co/Vishva007) | **Updated**: November 2025 |
|
|
|
|
|
|
|
|
|
