README.md

---

license: mit
base_model: microsoft/Phi-3-mini-4k-instruct
tags:
- phi-3
- lora
- payments
- finance
- information-extraction
- structured-data-extraction
- text-to-data
- finetuned
datasets:
- custom
language:
- en
pipeline_tag: text-generation
library_name: transformers
---


# Phi-3 Mini Reverse Fine-tuned for Payments Domain

This is a **reverse** fine-tuned version of [Microsoft's Phi-3-Mini-4k-Instruct](microsoft/Phi-3-mini-4k-instruct) model, adapted for extracting structured payment metadata from natural language descriptions using LoRA (Low-Rank Adaptation).

## Model Description

This model converts natural language payment descriptions into structured, machine-readable metadata. It performs the **opposite** task of the forward model - instead of generating human-friendly text, it extracts structured data that can be processed by payment APIs and applications.

### Related Models

**Forward Model (Companion):** [aamanlamba/phi3-payments-finetune](https://huggingface.co/aamanlamba/phi3-payments-finetune)
- Converts structured metadata → natural language
- Use together for round-trip validation

### Training Data

The model was trained on a dataset of 500+ synthetic payment transactions where:
- **Input**: Natural language payment descriptions
- **Output**: Structured metadata in `action(field[value], ...)` format

Transaction types covered:
- Standard payments (ACH, wire transfer, credit/debit card)
- Refunds (full and partial)
- Chargebacks and disputes
- Failed/declined transactions
- International transfers with currency conversion
- Transaction fees
- Recurring payments/subscriptions

### Example Usage

```python

from transformers import AutoModelForCausalLM, AutoTokenizer

from peft import PeftModel

import torch



# Load base model

base_model = "microsoft/Phi-3-mini-4k-instruct"

model = AutoModelForCausalLM.from_pretrained(

    base_model,

    torch_dtype=torch.float16,

    device_map="auto"

)



# Load LoRA adapters (reverse model)

model = PeftModel.from_pretrained(model, "aamanlamba/phi3-payments-reverse-finetune")

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)



# Extract structured data

prompt = """<|system|>

You are a financial data extraction assistant that converts natural language payment descriptions into structured metadata that can be processed by payment applications.<|end|>

<|user|>

Extract structured payment information from the following description:



Your payment of USD 1,500.00 to Global Supplies Inc via wire transfer was successfully completed on 2024-10-27.<|end|>

<|assistant|>

"""



inputs = tokenizer(prompt, return_tensors="pt").to(model.device)



with torch.no_grad():

    outputs = model.generate(

        **inputs,

        max_new_tokens=200,

        temperature=0.3,  # Lower temperature for more deterministic extraction

        top_p=0.9,

        do_sample=True

    )



response = tokenizer.decode(outputs[0], skip_special_tokens=True)

structured_data = response.split("<|assistant|>")[-1].strip()

print(structured_data)

```

**Expected output:**
```

inform(transaction_type[payment], amount[1500.00], currency[USD], receiver[Global Supplies Inc], status[completed], method[wire_transfer], date[2024-10-27])

```

### Parsing the Output

```python

import re



def parse_structured_data(structured_str: str) -> dict:

    """Parse structured payment data into a dictionary"""

    action_match = re.match(r'(\w+)\((.*)\)', structured_str)

    if not action_match:

        return None



    action_type = action_match.group(1)

    fields_str = action_match.group(2)



    fields = {}

    field_pattern = r'(\w+)\[(.*?)\]'

    for match in re.finditer(field_pattern, fields_str):

        field_name = match.group(1)

        field_value = match.group(2)



        # Convert numeric values

        if field_name in ['amount', 'refund_amount', 'fee_amount', 'exchange_rate']:

            try:

                field_value = float(field_value)

            except ValueError:

                pass



        fields[field_name] = field_value



    return {

        'action_type': action_type,

        'fields': fields

    }



# Use it

parsed = parse_structured_data(structured_data)

print(parsed)

# Output: {'action_type': 'inform', 'fields': {'transaction_type': 'payment', 'amount': 1500.0, ...}}

```

## Training Details

### Training Configuration

- **Base Model**: microsoft/Phi-3-mini-4k-instruct
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **Task Direction**: Natural Language → Structured Data (Reverse)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

- **Quantization**: 8-bit (training), float16 (inference)

- **Training Epochs**: 3

- **Learning Rate**: 2e-4

- **Batch Size**: 1 (with 8 gradient accumulation steps)

- **Hardware**: NVIDIA RTX 3060 (12GB VRAM)

- **Training Time**: ~35-45 minutes



### Training Loss



- Initial Loss: ~3.5-4.0

- Final Loss: ~0.8-1.2

- Validation Loss: ~1.0-1.3

- Extraction Accuracy: ~90-95% on validation set



## Model Size



- **LoRA Adapter Size**: ~15MB (only the adapter weights, not the full model)

- **Full Model Size**: ~7GB (when combined with base model)



## Supported Transaction Types



1. **Payments**: Standard payment transactions with various methods

2. **Refunds**: Full and partial refunds

3. **Chargebacks**: Dispute and chargeback processing

4. **Failed Payments**: Declined or failed transactions with reasons

5. **International Transfers**: Cross-border payments with currency conversion

6. **Fees**: Transaction and processing fees

7. **Recurring Payments**: Subscriptions and scheduled payments

8. **Reversals**: Payment reversals and adjustments



## Output Format



The model extracts data in this structured format:

```

action_type(field1[value1], field2[value2], ...)
```



**Action Types:**

- `inform`: Informational transactions (payments, refunds, transfers)

- `alert`: Alerts and notifications (failures, chargebacks)



**Common Fields:**

- `transaction_type`: Type of transaction

- `amount`: Transaction amount (numeric)

- `currency`: Currency code (USD, EUR, GBP, etc.)

- `sender`/`receiver`/`merchant`: Party names

- `status`: Transaction status (completed, pending, failed, etc.)

- `method`: Payment method (credit_card, ACH, wire_transfer, etc.)

- `date`: Transaction date (YYYY-MM-DD)

- `reason`: Failure/chargeback reason (for alerts)



## Use Cases



### 1. Conversational Payment Interfaces

Extract payment details from user messages:

```
User: "I want to send $500 to John via PayPal"
Extracted: inform(transaction_type[payment], amount[500], currency[USD], receiver[John], method[PayPal])

```



### 2. Email Parsing

Extract transaction data from payment notification emails automatically.



### 3. Voice Payment Systems

Convert spoken payment descriptions into structured API calls.



### 4. Payment API Integration

Transform natural language payment requests into API-ready parameters.



## Limitations



- Trained on synthetic data - may require additional fine-tuning for production use

- Optimized for English language only

- Best performance on transaction patterns similar to training data

- Output format is custom - requires parsing (see example above)

- Not suitable for handling real financial transactions without validation

- Lower temperature (0.3) recommended for consistent extraction



## Ethical Considerations



- This model was trained on synthetic, anonymized data only

- Does not contain any real customer PII or transaction data

- Should be validated for accuracy before production deployment

- Implement validation and error handling for extracted data

- Consider regulatory compliance (PCI-DSS, GDPR, etc.) in your jurisdiction

- Always verify extracted financial data before processing



## Intended Use



**Primary Use Cases:**

- Extracting transaction data from natural language descriptions

- Building conversational payment bots

- Parsing payment notifications and emails

- Converting user requests to API parameters

- Training and demonstration purposes

- Research in financial NLP and information extraction



**Out of Scope:**

- Direct transaction processing without validation

- Real-time financial systems without error handling

- Compliance-critical data extraction

- Medical or legal payment processing



## Performance Notes



- **Inference Speed**: ~2-3 seconds per extraction on RTX 3060

- **Temperature**: Use 0.1-0.3 for deterministic extraction

- **Validation**: Always validate output format and field values

- **Error Handling**: Implement fallbacks for malformed outputs



## How to Cite



If you use this model in your research or application, please cite:



```bibtex

@misc{phi3-payments-reverse-finetuned,

  author = {aamanlamba},

  title = {Phi-3 Mini Reverse Fine-tuned for Payments Domain},

  year = {2024},

  publisher = {HuggingFace},

  howpublished = {\url{https://huggingface.co/aamanlamba/phi3-payments-reverse-finetune}}

}

```



## Training Code



The complete training code and dataset generation scripts are available on GitHub:

- **Repository**: [github.com/aamanlamba/phi3-tune-payments](https://github.com/aamanlamba/phi3-tune-payments)

- **Branch**: `reverse-structured-extraction` (this model)

- **Includes**: Reverse dataset generator, training scripts, testing utilities, parsing examples



## Acknowledgements



- Base model: [Microsoft Phi-3-Mini-4k-Instruct](microsoft/Phi-3-mini-4k-instruct)

- Fine-tuning method: [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)

- Training framework: HuggingFace Transformers + PEFT

- Inspired by: [NVIDIA AI Workbench Phi-3 Fine-tuning Example](https://github.com/NVIDIA/workbench-example-phi3-finetune)



## License



This model is released under the MIT license, compatible with the base Phi-3 model license.



## Contact



For questions or issues, please open an issue on the GitHub repository or contact the author.



---



**Note**: This is a **reverse** model for structured data extraction. For generating natural language from structured data, see the companion forward model.