|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- finance |
|
|
- volatility |
|
|
- time-series |
|
|
- forecasting |
|
|
- chronos |
|
|
- peft |
|
|
- lora |
|
|
base_model: amazon/chronos-t5-mini |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# chronos-volatility |
|
|
|
|
|
## Model Description |
|
|
|
|
|
|
|
|
A fine-tuned Chronos model for predicting stock market volatility. |
|
|
This model predicts quantiles (q10, q50, q90) of log-realized variance |
|
|
for a 20-day forward horizon using 60 days of historical squared returns. |
|
|
|
|
|
|
|
|
This model is a fine-tuned version of [amazon/chronos-t5-mini](https://huggingface.co/amazon/chronos-t5-mini) using LoRA (Low-Rank Adaptation) for volatility prediction on stock market data. |
|
|
|
|
|
## Model Architecture |
|
|
|
|
|
- **Base Model**: amazon/chronos-t5-mini |
|
|
- **Task**: Volatility Prediction (Quantile Regression) |
|
|
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation) |
|
|
- **LoRA Config**: r=8, alpha=16, target_modules=["q", "k", "v", "o"] |
|
|
- **Custom Heads**: |
|
|
- Value Embedding: Linear(1 -> hidden_dim) for embedding raw time series values |
|
|
- Quantile Head: Linear(hidden_dim -> 3) for predicting q10, q50, q90 quantiles |
|
|
|
|
|
## Input Format |
|
|
|
|
|
- **Input**: Single-channel time series of squared returns (shape: batch_size, seq_length=60) |
|
|
- **Sequence Length**: 60 trading days |
|
|
- **Preprocessing**: Input should be squared log returns: `(log(price_t / price_{t-1}))^2` |
|
|
|
|
|
## Output Format |
|
|
|
|
|
- **Output**: Quantiles in log-variance space (shape: batch_size, 3) |
|
|
- Column 0: q10 (10th percentile) |
|
|
- Column 1: q50 (50th percentile / median) |
|
|
- Column 2: q90 (90th percentile) |
|
|
|
|
|
To convert to annualized volatility: |
|
|
```python |
|
|
variance = np.exp(log_variance) |
|
|
volatility = np.sqrt(variance) * np.sqrt(252) # Annualized |
|
|
``` |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Loading the Model |
|
|
|
|
|
```python |
|
|
from src.models.chronos import ChronosVolatility |
|
|
import torch |
|
|
|
|
|
# Load model |
|
|
model = ChronosVolatility(use_lora=True) |
|
|
model.load_custom_heads("path/to/heads.pt") |
|
|
model.base.load_adapter("path/to/adapter") # For PEFT adapter |
|
|
|
|
|
# Or use from_pretrained (if properly saved): |
|
|
# from peft import PeftModel |
|
|
# base_model = AutoModelForSeq2SeqLM.from_pretrained("amazon/chronos-t5-mini") |
|
|
# model.base = PeftModel.from_pretrained(base_model, "chronos-volatility") |
|
|
# model.load_custom_heads("path/to/heads.pt") |
|
|
``` |
|
|
|
|
|
### Making Predictions |
|
|
|
|
|
```python |
|
|
import numpy as np |
|
|
|
|
|
# Prepare input: squared returns sequence (60 days) |
|
|
input_seq = torch.FloatTensor(squared_returns).unsqueeze(0) # (1, 60) |
|
|
|
|
|
# Get predictions |
|
|
model.eval() |
|
|
with torch.no_grad(): |
|
|
quantiles_log_var = model(input_seq) # (1, 3) |
|
|
|
|
|
# Convert to volatility |
|
|
quantiles_var = np.exp(quantiles_log_var.numpy()) |
|
|
quantiles_vol = np.sqrt(quantiles_var) * np.sqrt(252) # Annualized % |
|
|
|
|
|
print(f"10th percentile: {quantiles_vol[0][0]:.2f}%") |
|
|
print(f"Median: {quantiles_vol[0][1]:.2f}%") |
|
|
print(f"90th percentile: {quantiles_vol[0][2]:.2f}%") |
|
|
``` |
|
|
|
|
|
## Training Details |
|
|
- Trained data from AAPL, GOOG, MSFT, SPY, and TSLA |
|
|
- NVDA was used as a test case |
|
|
|
|
|
## Limitations |
|
|
- Trained on large-cap tech stocks, and will not generalize well to other markets or time periods. |
|
|
- Predictions are probabilistic (quantiles) and should be interpreted with uncertainty |
|
|
- The model requires at least 60 days of historical data for predictions |
|
|
|
|
|
|
