---
license: mit
base_model: microsoft/Phi-3-mini-4k-instruct
tags:
- phi-3
- lora
- payments
- finance
- natural-language-generation
- finetuned
datasets:
- custom
language:
- en
pipeline_tag: text-generation
library_name: transformers
---

# Phi-3 Mini Fine-tuned for Payments Domain

This is a fine-tuned version of [Microsoft's Phi-3-Mini-4k-Instruct](microsoft/Phi-3-mini-4k-instruct) model, adapted for generating natural language descriptions of payment transactions using LoRA (Low-Rank Adaptation).

## Model Description

This model converts structured payment transaction data into clear, customer-friendly language. It was fine-tuned using LoRA on a synthetic payments dataset covering various transaction types.

### Training Data

The model was trained on a dataset of 500+ synthetic payment transactions including:
- Standard payments (ACH, wire transfer, credit/debit card)
- Refunds (full and partial)
- Chargebacks
- Failed/declined transactions
- International transfers with currency conversion
- Transaction fees
- Recurring payments/subscriptions

### Example Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype="auto",
    device_map="auto"
)

# Load LoRA adapters
model = PeftModel.from_pretrained(model, "aamanlamba/phi3-payments-finetune")
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Generate description
prompt = """<|system|>
You are a financial services assistant that explains payment transactions in clear, customer-friendly language.<|end|>
<|user|>
Convert the following structured payment information into a natural explanation:

inform(transaction_type[payment], amount[1500.00], currency[USD], sender[Acme Corp], receiver[Global Supplies Inc], status[completed], method[ACH], date[2024-10-27])<|end|>
<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

Expected output:
```
Your ACH payment of $1,500.00 to Global Supplies Inc was successfully completed on October 27, 2024.
```

## Training Details

### Training Configuration

- **Base Model**: microsoft/Phi-3-mini-4k-instruct
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Quantization**: 8-bit (training), float16 (inference)
- **Training Epochs**: 3
- **Learning Rate**: 2e-4
- **Batch Size**: 1 (with 8 gradient accumulation steps)
- **Hardware**: NVIDIA RTX 3060 (12GB VRAM)
- **Training Time**: ~35-45 minutes

### Training Loss

- Initial Loss: ~3.5-4.0
- Final Loss: ~0.9-1.2
- Validation Loss: ~1.0-1.3

## Model Size

- **LoRA Adapter Size**: ~15MB (only the adapter weights, not the full model)
- **Full Model Size**: ~7GB (when combined with base model)

## Supported Transaction Types

1. **Payments**: Standard payment transactions
2. **Refunds**: Full and partial refunds
3. **Chargebacks**: Dispute and chargeback processing
4. **Failed Payments**: Declined or failed transactions with reasons
5. **International Transfers**: Cross-border payments with currency conversion
6. **Fees**: Transaction and processing fees
7. **Recurring Payments**: Subscriptions and scheduled payments
8. **Reversals**: Payment reversals and adjustments

## Limitations

- Trained on synthetic data - may require additional fine-tuning for production use
- Optimized for English language only
- Best performance on transaction patterns similar to training data
- Not suitable for handling real financial transactions without human oversight
- Should not be used as the sole system for financial communication

## Ethical Considerations

- This model was trained on synthetic, anonymized data only
- Does not contain any real customer PII or transaction data
- Should be validated for accuracy before production deployment
- Implement human review for customer-facing financial communications
- Consider regulatory compliance (PCI-DSS, GDPR, etc.) in your jurisdiction

## Intended Use

**Primary Use Cases:**
- Generating transaction descriptions for internal systems
- Creating customer-friendly payment notifications
- Automating payment communication drafts (with human review)
- Training and demonstration purposes
- Research in financial NLP

**Out of Scope:**
- Direct customer communication without review
- Real-time transaction processing without validation
- Compliance-critical communications
- Medical or legal payment descriptions

## How to Cite

If you use this model in your research or application, please cite:

```bibtex
@misc{phi3-payments-finetuned,
  author = {aamanlamba},
  title = {Phi-3 Mini Fine-tuned for Payments Domain},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/aamanlamba/phi3-payments-finetune}}
}
```

## Training Code

The complete training code and dataset generation scripts are available on GitHub:
- **Repository**: [github.com/aamanlamba/phi3-tune-payments](https://github.com/aamanlamba/phi3-tune-payments)
- **Includes**: Dataset generator, training scripts, testing utilities, and deployment guides

## Acknowledgements

- Base model: [Microsoft Phi-3-Mini-4k-Instruct](microsoft/Phi-3-mini-4k-instruct)
- Fine-tuning method: [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)
- Training framework: HuggingFace Transformers + PEFT
- Inspired by: [NVIDIA AI Workbench Phi-3 Fine-tuning Example](https://github.com/NVIDIA/workbench-example-phi3-finetune)

## License

This model is released under the MIT license, compatible with the base Phi-3 model license.

## Contact

For questions or issues, please open an issue on the model repository or contact the author.

---

**Note**: This is a demonstration model. Always validate outputs for accuracy before use in production financial systems.