File size: 6,325 Bytes
85d2aa9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 |
---
license: mit
base_model: microsoft/Phi-3-mini-4k-instruct
tags:
- phi-3
- lora
- payments
- finance
- natural-language-generation
- finetuned
datasets:
- custom
language:
- en
pipeline_tag: text-generation
library_name: transformers
---
# Phi-3 Mini Fine-tuned for Payments Domain
This is a fine-tuned version of [Microsoft's Phi-3-Mini-4k-Instruct](microsoft/Phi-3-mini-4k-instruct) model, adapted for generating natural language descriptions of payment transactions using LoRA (Low-Rank Adaptation).
## Model Description
This model converts structured payment transaction data into clear, customer-friendly language. It was fine-tuned using LoRA on a synthetic payments dataset covering various transaction types.
### Training Data
The model was trained on a dataset of 500+ synthetic payment transactions including:
- Standard payments (ACH, wire transfer, credit/debit card)
- Refunds (full and partial)
- Chargebacks
- Failed/declined transactions
- International transfers with currency conversion
- Transaction fees
- Recurring payments/subscriptions
### Example Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype="auto",
device_map="auto"
)
# Load LoRA adapters
model = PeftModel.from_pretrained(model, "aamanlamba/phi3-payments-finetune")
tokenizer = AutoTokenizer.from_pretrained(base_model)
# Generate description
prompt = """<|system|>
You are a financial services assistant that explains payment transactions in clear, customer-friendly language.<|end|>
<|user|>
Convert the following structured payment information into a natural explanation:
inform(transaction_type[payment], amount[1500.00], currency[USD], sender[Acme Corp], receiver[Global Supplies Inc], status[completed], method[ACH], date[2024-10-27])<|end|>
<|assistant|>
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```
Expected output:
```
Your ACH payment of $1,500.00 to Global Supplies Inc was successfully completed on October 27, 2024.
```
## Training Details
### Training Configuration
- **Base Model**: microsoft/Phi-3-mini-4k-instruct
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Quantization**: 8-bit (training), float16 (inference)
- **Training Epochs**: 3
- **Learning Rate**: 2e-4
- **Batch Size**: 1 (with 8 gradient accumulation steps)
- **Hardware**: NVIDIA RTX 3060 (12GB VRAM)
- **Training Time**: ~35-45 minutes
### Training Loss
- Initial Loss: ~3.5-4.0
- Final Loss: ~0.9-1.2
- Validation Loss: ~1.0-1.3
## Model Size
- **LoRA Adapter Size**: ~15MB (only the adapter weights, not the full model)
- **Full Model Size**: ~7GB (when combined with base model)
## Supported Transaction Types
1. **Payments**: Standard payment transactions
2. **Refunds**: Full and partial refunds
3. **Chargebacks**: Dispute and chargeback processing
4. **Failed Payments**: Declined or failed transactions with reasons
5. **International Transfers**: Cross-border payments with currency conversion
6. **Fees**: Transaction and processing fees
7. **Recurring Payments**: Subscriptions and scheduled payments
8. **Reversals**: Payment reversals and adjustments
## Limitations
- Trained on synthetic data - may require additional fine-tuning for production use
- Optimized for English language only
- Best performance on transaction patterns similar to training data
- Not suitable for handling real financial transactions without human oversight
- Should not be used as the sole system for financial communication
## Ethical Considerations
- This model was trained on synthetic, anonymized data only
- Does not contain any real customer PII or transaction data
- Should be validated for accuracy before production deployment
- Implement human review for customer-facing financial communications
- Consider regulatory compliance (PCI-DSS, GDPR, etc.) in your jurisdiction
## Intended Use
**Primary Use Cases:**
- Generating transaction descriptions for internal systems
- Creating customer-friendly payment notifications
- Automating payment communication drafts (with human review)
- Training and demonstration purposes
- Research in financial NLP
**Out of Scope:**
- Direct customer communication without review
- Real-time transaction processing without validation
- Compliance-critical communications
- Medical or legal payment descriptions
## How to Cite
If you use this model in your research or application, please cite:
```bibtex
@misc{phi3-payments-finetuned,
author = {aamanlamba},
title = {Phi-3 Mini Fine-tuned for Payments Domain},
year = {2024},
publisher = {HuggingFace},
howpublished = {\url{https://huggingface.co/aamanlamba/phi3-payments-finetune}}
}
```
## Training Code
The complete training code and dataset generation scripts are available on GitHub:
- **Repository**: [github.com/aamanlamba/phi3-tune-payments](https://github.com/aamanlamba/phi3-tune-payments)
- **Includes**: Dataset generator, training scripts, testing utilities, and deployment guides
## Acknowledgements
- Base model: [Microsoft Phi-3-Mini-4k-Instruct](microsoft/Phi-3-mini-4k-instruct)
- Fine-tuning method: [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)
- Training framework: HuggingFace Transformers + PEFT
- Inspired by: [NVIDIA AI Workbench Phi-3 Fine-tuning Example](https://github.com/NVIDIA/workbench-example-phi3-finetune)
## License
This model is released under the MIT license, compatible with the base Phi-3 model license.
## Contact
For questions or issues, please open an issue on the model repository or contact the author.
---
**Note**: This is a demonstration model. Always validate outputs for accuracy before use in production financial systems.
|