README.md

---
language:
- en
license: apache-2.0
tags:
- finance
- options-trading
- market-prediction
- quantitative-analysis
- qlora
- mistral
base_model: mistralai/Mistral-7B-Instruct-v0.3
datasets:
- sujet-ai/Sujet-Finance-Instruct-177k
- gbharti/finance-alpaca
- Josephgflowers/Finance-Instruct-500k
pipeline_tag: text-generation
---

# 📈 FinOptions-Mistral-7B — Options & Market Prediction Expert

A **Mistral-7B-Instruct** model fine-tuned with **QLoRA** on **~745K financial instruction examples** to be an expert at:

1. **Options trading analysis** — strategies, Greeks, implied volatility, risk management
2. **Explaining HOW data features affect market predictions** — feature importance, directional impact
3. **Step-by-step market reasoning** — breaking down complex financial scenarios with quantitative logic

## 🧠 What Makes This Model Special

Unlike generic financial models, this one is trained with a system prompt that forces **interpretable, step-by-step reasoning**:

> For every analysis: (1) Identify which data features are most influential, (2) Explain the directional impact of each, (3) Provide your strategy recommendation with reasoning, (4) Express confidence and risk factors.

This means when you ask about options or market movements, you get **explanations of WHY**, not just predictions.

## 📊 Training Details

| Parameter | Value |
|---|---|
| **Base Model** | `mistralai/Mistral-7B-Instruct-v0.3` |
| **Method** | QLoRA (4-bit NF4 + double quantization) |
| **LoRA Rank** | 64 |
| **LoRA Alpha** | 128 |
| **Target Modules** | All linear layers (q/k/v/o/gate/up/down_proj) |
| **Learning Rate** | 2e-4 (cosine schedule) |
| **Epochs** | 2 |
| **Effective Batch Size** | 16 (2 × 8 gradient accumulation) |
| **Max Sequence Length** | 2048 |
| **Optimizer** | paged_adamw_8bit |
| **Precision** | bf16 |

### Training Recipe Source
Based on the **Open-FinLLMs** paper (arxiv:2408.11878) which achieved:
- Outperformed GPT-4 on 14 financial tasks across 30 datasets
- TSLA cumulative trading return: **0.56** vs -0.18 (Buy & Hold)
- QLoRA r=64, α=128 on all linear modules (verified to match full fine-tune quality)

## 📚 Training Data (~745K examples)

| Dataset | Size | Content |
|---|---|---|
| [sujet-ai/Sujet-Finance-Instruct-177k](https://huggingface.co/datasets/sujet-ai/Sujet-Finance-Instruct-177k) | 177K | Sentiment analysis, NER, financial Q&A, classification |
| [gbharti/finance-alpaca](https://huggingface.co/datasets/gbharti/finance-alpaca) | 68K | Financial Q&A (includes options, investing, markets) |
| [Josephgflowers/Finance-Instruct-500k](https://huggingface.co/datasets/Josephgflowers/Finance-Instruct-500k) | 500K | Broad financial instruction-following |

All datasets converted to ChatML `messages` format with financial expert system prompts.

## 🚀 Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model + LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "Saksham7772/FinOptions-Mistral-7B")
tokenizer = AutoTokenizer.from_pretrained("Saksham7772/FinOptions-Mistral-7B")

# Ask about options / market prediction
messages = [
    {"role": "user", "content": """
    AAPL is trading at $185. Earnings are in 5 days.
    IV Rank is at 82%, Put/Call ratio is 1.3, and the stock dropped 2.5% today.
    RSI is at 35. What options strategy would you recommend and why?
    Which of these data points matter most for the prediction?
    """}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### Example Queries

- *"How does implied volatility affect options pricing, and what should I do when IV rank is above 80%?"*
- *"Given these features: volume spike (+300%), earnings in 3 days, RSI oversold, and positive sentiment shift — what's the likely market direction and which feature matters most?"*
- *"Explain a bull call spread strategy. When should I use it vs a simple long call?"*
- *"If the Fed raises rates by 25bp, how would that affect tech sector options? Walk me through each factor."*

## 🏋️ Train It Yourself

The complete training script is included in this repo: [`train.py`](./train.py)

```bash
# Install dependencies
pip install torch transformers trl peft bitsandbytes datasets trackio accelerate

# Set your HF token
export HF_TOKEN=your_token_here

# Run training (requires GPU with 24GB+ VRAM, e.g. A100/A10G)
python train.py
```

**Hardware requirements:**
- **Minimum**: 1× A10G (24GB) — will work with batch_size=1
- **Recommended**: 1× A100 (80GB) — comfortable with batch_size=2
- **Training time**: ~6-8 hours on A100

## 📖 Research Background

This model draws from several key papers in financial LLMs:

1. **Open-FinLLMs** (2408.11878) — The primary training recipe. Found that 3:1 financial:general data ratio prevents catastrophic forgetting, and including math instruction data (MathInstruct) significantly improves numerical reasoning.

2. **FinTral** (2402.10986) — Key insight for interpretability: "memetic proxy" prompting (casting the model as a financial expert) + step-by-step constraints produces the most interpretable outputs. Mistral's digit-level BPE tokenizer handles financial numbers better than LLaMA.

3. **FinGPT** (2306.06031) — Showed that LoRA SFT achieves 77.3 Macro-F1 on financial sentiment (vs FinBERT 69.9, GPT-4 zero-shot 61.7), and RLSP alignment can push this to 82.1.

4. **Harnessing Earnings Reports** (2408.06634) — Demonstrated that **textualizing numerical data** into descriptive sentences (rather than raw numbers) dramatically improves LLM financial reasoning.

## ⚠️ Limitations

- This is a **fine-tuned language model**, NOT a trading bot. It generates text-based analysis, not trading signals.
- Financial markets are inherently unpredictable. Model outputs should be used as one input among many in investment decisions.
- The training data is from public datasets and may not reflect the most current market conditions.
- No options-specific pricing dataset exists publicly — the model learns options concepts from Q&A data, not from options chain data directly.
- **Not financial advice.** Always consult qualified financial professionals.

## 📝 License

Apache 2.0 (same as base Mistral model)