📈 FinOptions-Mistral-7B — Options & Market Prediction Expert

A Mistral-7B-Instruct model fine-tuned with QLoRA on ~745K financial instruction examples to be an expert at:

1. Options trading analysis — strategies, Greeks, implied volatility, risk management
2. Explaining HOW data features affect market predictions — feature importance, directional impact
3. Step-by-step market reasoning — breaking down complex financial scenarios with quantitative logic

🧠 What Makes This Model Special

Unlike generic financial models, this one is trained with a system prompt that forces interpretable, step-by-step reasoning:

For every analysis: (1) Identify which data features are most influential, (2) Explain the directional impact of each, (3) Provide your strategy recommendation with reasoning, (4) Express confidence and risk factors.

This means when you ask about options or market movements, you get explanations of WHY, not just predictions.

📊 Training Details

Parameter	Value
Base Model	mistralai/Mistral-7B-Instruct-v0.3
Method	QLoRA (4-bit NF4 + double quantization)
LoRA Rank	64
LoRA Alpha	128
Target Modules	All linear layers (q/k/v/o/gate/up/down_proj)
Learning Rate	2e-4 (cosine schedule)
Epochs	2
Effective Batch Size	16 (2 × 8 gradient accumulation)
Max Sequence Length	2048
Optimizer	paged_adamw_8bit
Precision	bf16

Training Recipe Source

Based on the Open-FinLLMs paper (arxiv:2408.11878) which achieved:

• Outperformed GPT-4 on 14 financial tasks across 30 datasets
• TSLA cumulative trading return: 0.56 vs -0.18 (Buy & Hold)
• QLoRA r=64, α=128 on all linear modules (verified to match full fine-tune quality)

📚 Training Data (~745K examples)

Dataset	Size	Content
sujet-ai/Sujet-Finance-Instruct-177k	177K	Sentiment analysis, NER, financial Q&A, classification
gbharti/finance-alpaca	68K	Financial Q&A (includes options, investing, markets)
Josephgflowers/Finance-Instruct-500k	500K	Broad financial instruction-following

All datasets converted to ChatML messages format with financial expert system prompts.

🚀 Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model + LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "Saksham7772/FinOptions-Mistral-7B")
tokenizer = AutoTokenizer.from_pretrained("Saksham7772/FinOptions-Mistral-7B")

# Ask about options / market prediction
messages = [
    {"role": "user", "content": """
    AAPL is trading at $185. Earnings are in 5 days.
    IV Rank is at 82%, Put/Call ratio is 1.3, and the stock dropped 2.5% today.
    RSI is at 35. What options strategy would you recommend and why?
    Which of these data points matter most for the prediction?
    """}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example Queries

• "How does implied volatility affect options pricing, and what should I do when IV rank is above 80%?"
• "Given these features: volume spike (+300%), earnings in 3 days, RSI oversold, and positive sentiment shift — what's the likely market direction and which feature matters most?"
• "Explain a bull call spread strategy. When should I use it vs a simple long call?"
• "If the Fed raises rates by 25bp, how would that affect tech sector options? Walk me through each factor."

🏋️ Train It Yourself

The complete training script is included in this repo: train.py

# Install dependencies
pip install torch transformers trl peft bitsandbytes datasets trackio accelerate

# Set your HF token
export HF_TOKEN=your_token_here

# Run training (requires GPU with 24GB+ VRAM, e.g. A100/A10G)
python train.py

Hardware requirements:

• Minimum: 1× A10G (24GB) — will work with batch_size=1
• Recommended: 1× A100 (80GB) — comfortable with batch_size=2
• Training time: ~6-8 hours on A100

📖 Research Background

This model draws from several key papers in financial LLMs:

1. Open-FinLLMs (2408.11878) — The primary training recipe. Found that 3:1 financial:general data ratio prevents catastrophic forgetting, and including math instruction data (MathInstruct) significantly improves numerical reasoning.

2. FinTral (2402.10986) — Key insight for interpretability: "memetic proxy" prompting (casting the model as a financial expert) + step-by-step constraints produces the most interpretable outputs. Mistral's digit-level BPE tokenizer handles financial numbers better than LLaMA.

3. FinGPT (2306.06031) — Showed that LoRA SFT achieves 77.3 Macro-F1 on financial sentiment (vs FinBERT 69.9, GPT-4 zero-shot 61.7), and RLSP alignment can push this to 82.1.

4. Harnessing Earnings Reports (2408.06634) — Demonstrated that textualizing numerical data into descriptive sentences (rather than raw numbers) dramatically improves LLM financial reasoning.

⚠️ Limitations

• This is a fine-tuned language model, NOT a trading bot. It generates text-based analysis, not trading signals.
• Financial markets are inherently unpredictable. Model outputs should be used as one input among many in investment decisions.
• The training data is from public datasets and may not reflect the most current market conditions.
• No options-specific pricing dataset exists publicly — the model learns options concepts from Q&A data, not from options chain data directly.
• Not financial advice. Always consult qualified financial professionals.

📝 License

Apache 2.0 (same as base Mistral model)