README.md

---
language: en
tags:
- financial-analysis
- covenant-extraction
- llama
- lora
license: llama2
datasets:
- custom_financial_covenants
metrics:
- accuracy
pipeline_tag: text-generation
inference: true
library_name: transformers
widget:
  - text: |
      ### Instruction: Extract covenant details from the following credit agreement section and structure it into JSON format only.

      ### Input: Section 4.2:
      The Borrower shall maintain a Fixed Charge Coverage Ratio of not less than 1.25:1.00 for any fiscal quarter ending after June 30, 2024.

      ### Response:
model-index:
- name: covenant-extractor
  results:
  - task:
      type: text2json
      name: Financial Covenant Extraction
    metrics:
      - type: accuracy
        value: 90.0
        name: Test Accuracy
---

# Covenant Extractor Model

This model is fine-tuned on Llama-3.2-3B-Instruct for extracting and structuring financial covenants from credit agreements into standardized JSON format.

## Model Description

- **Base Model:** meta-llama/Llama-3.2-3B-Instruct
- **Task:** Financial Covenant Extraction
- **Training Method:** LoRA Fine-tuning
- **Language:** English
- **License:** Same as base model

## Intended Use

This model is designed to:
- Extract covenant details from credit agreement sections
- Structure the information into standardized JSON format
- Handle various types of financial covenants (leverage ratios, coverage ratios, etc.)

## Input Format

```
### Instruction: Extract covenant details from the following credit agreement section and structure it into JSON format only.

### Input: Section 4.2:
The Borrower shall maintain a Fixed Charge Coverage Ratio of not less than 1.25:1.00 for any fiscal quarter ending after June 30, 2024.

### Response:
```

## Output Format

```json
{
    "type": "financial",
    "category": "fixed_charge_coverage_ratio",
    "section": "4.2",
    "requirements": {
        "threshold": "1.25:1.00",
        "measurement_period": "quarterly",
        "timeline": ["June 30, 2024"]
    }
}
```

## Training Details

- **Training Method:** LoRA (Low-Rank Adaptation)
- **LoRA Config:**
  - Rank: 16
  - Alpha: 32
  - Target Modules: q_proj, k_proj, v_proj, o_proj
  - Dropout: 0.1
- **Training Parameters:**
  - Batch Size: 4
  - Gradient Accumulation Steps: 16
  - Learning Rate: 1e-4
  - Number of Epochs: 3
  - Weight Decay: 0.01
  - Max Gradient Norm: 1.0

## Limitations

- Only processes English language credit agreements
- Best suited for standard financial covenants
- May require adjustment for complex or non-standard covenant structures

## Citation

If you use this model in your work, please cite:
```
@misc{covenant-extractor,
  author = {[Bikram Adhikari]},
  title = {Covenant Extractor: Fine-tuned LLM for Financial Covenant Analysis},
  year = {2024}
}
```