README.md

---
tags:
- time-series-prediction
- transformer
- cryptocurrency
- finance
datasets:
- crypto-historical-data
license: mit
---

# CryptoTS-Transformer-Forecast-7D

## 📝 Overview

**CryptoTS-Transformer-Forecast-7D** is a purpose-built Transformer model designed for financial time series forecasting, specifically predicting the **next 7 days of the closing price** for major cryptocurrencies (e.g., BTC, ETH).

Unlike traditional RNNs, this architecture leverages the attention mechanism to capture long-range dependencies and complex correlations across multiple technical indicators, leading to more robust sequence-to-sequence predictions.

## 🧠 Model Architecture

The model is a custom encoder-decoder Transformer architecture:

* **Type:** Sequence-to-Sequence Time Series Transformer.
* **Input Sequence Length (`seq_len`):** 60 time steps (representing 60 days of historical data).
* **Output Sequence Length (`output_len`):** 7 time steps (the 7 days being predicted).
* **Input Features (`input_size`):** 12 features, including OHLCV data and 7 key technical indicators (MACD, RSI, various moving averages and Bollinger Bands).
* **Core Components:**
* **Positional Encoding:** Added to input embeddings to preserve the temporal order.
* **Multi-Head Attention:** Utilized within both the encoder and decoder to weigh the importance of different time steps and features.
* **Loss Function:** Mean Squared Error (MSE).

## 🚀 Intended Use

* **Financial Forecasting:** Generating forward-looking predictions for high-volatility financial assets.
* **Algorithmic Trading Signal:** Use the 7-day forecast as input for automated trading strategies.
* **Risk Assessment:** Modeling potential near-term price volatility based on predicted ranges.

## ⚠️ Limitations

* **Non-Stationarity:** Financial time series are inherently non-stationary. While the model uses robust scaling, sudden market shifts (black swan events, major news) can lead to significant forecast errors.
* **Data Quality:** The model's performance is heavily reliant on the accuracy and quality of the input features, particularly the technical indicators.
* **Overfitting:** Due to the complexity of the Transformer, there is a risk of overfitting to historical noise patterns.

## 💻 Example Code

This example assumes a pre-processed input tensor `input_data` (60 days of 12 features) scaled using a standard scaler.

```python
import torch
import torch.nn as nn
from transformers import AutoModel

# NOTE: This model uses a custom class not available in standard transformers,
# but can be loaded via a custom class or registered in AutoModel.
# For demonstration, we assume it's loaded as a generic AutoModel.
model_name = "Your-HF-Username/CryptoTS-Transformer-Forecast-7D"
model = AutoModel.from_pretrained(model_name)
model.eval()

# Dummy input data: (Batch_Size, Sequence_Length, Input_Features) -> (1, 60, 12)
# In a real scenario, this would be 60 days of OHLCV + 7 Indicators, scaled.
input_data = torch.randn(1, model.config.seq_len, model.config.input_size)

# Make prediction
with torch.no_grad():
# Output shape will be (Batch_Size, Output_Length, 1) or (1, 7, 1) for the predicted Close price
prediction = model(input_data)

# Convert predicted prices back to their original scale using the inverse transform
# predicted_prices_7d = scaler.inverse_transform(prediction.cpu().numpy())

print("Input shape (60 days, 12 features):", input_data.shape)
print("Predicted output shape (7 days, 1 feature/price):", prediction.shape)
print("Predicted 7-day normalized closing prices:\n", prediction.squeeze().numpy())

config.json

@@ -0,0 +1,28 @@
+{
+  "architectures": [
+    "TransformerForTimeSeriesPrediction"
+  ],
+  "model_type": "time-series-transformer",
+  "input_size": 12,
+  "hidden_size": 128,
+  "num_layers": 4,
+  "num_heads": 8,
+  "dropout": 0.1,
+  "seq_len": 60,
+  "output_len": 7,
+  "feature_columns": [
+    "Open",
+    "High",
+    "Low",
+    "Close",
+    "Volume",
+    "MarketCap",
+    "MACD",
+    "RSI",
+    "Bollinger_H",
+    "Bollinger_L",
+    "EMA_20",
+    "SMA_50"
+  ],
+  "scaling_method": "standard_scaler"
+}