checkpoint/python_version/GUIDE.md

# ML-3m-trader Usage Guide

Step-by-step instructions for running the full ML trading pipeline.

---

## Prerequisites

| Requirement | Details |
|-------------|---------|
| Python | 3.9 or later |
| MetaTrader 5 | Installed, running, and logged in to a broker account with XAUUSDc |
| Operating System | Windows (required by the MetaTrader5 Python API) |

---

## 1. Environment Setup

Create a virtual environment and install dependencies:

```
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
```

---

## 2. Available Commands

| Command | Description |
|---------|-------------|
| `python main.py fetch` | Connect to MT5 and download 1 year of 3-minute OHLCV data for XAUUSDc |
| `python main.py train` | Compute features, generate labels, and train the LightGBM model |
| `python main.py backtest` | Run a walk-forward backtest on the held-out test set |
| `python main.py evaluate` | Re-print the saved backtest performance report |
| `python main.py predict` | Show latest class probabilities from the trained model |
| `python main.py run` | Full pipeline in one command: fetch, train, backtest, evaluate |

---

## 3. Step-by-Step Workflow

### 3.1 Fetch Data

1. Open MetaTrader 5 and log in to your broker account.
2. Ensure the symbol **XAUUSDc** is visible in Market Watch.
3. Run:

```
python main.py fetch
```

**Output**: A CSV file is saved to `data/raw_xauusdc_m3.csv`.

| Output Column | Description |
|---------------|-------------|
| time | Bar open time (UTC) |
| open | Opening price |
| high | Highest price in the bar |
| low | Lowest price in the bar |
| close | Closing price |
| volume | Tick volume |
| spread | Broker spread (in points) |

---

### 3.2 Train the Model

```
python main.py train
```

This command performs three stages:

| Stage | Action |
|-------|--------|
| Feature Engineering | Computes SMA, Double MA, VROC, Synthetic VIX, MSI, ADX, and cyclical time features |
| Label Generation | Simulates 1:1 RR trades at each bar to classify: BUY, SELL, HOLD, DO_NOTHING |
| Model Training | Trains a LightGBM multi-class classifier with chronological 80/20 split and early stopping |

**Output**:
- Featured dataset saved to `data/featured_data.csv`
- Trained model saved to `models/lgbm_model.pkl`
- Feature importance printed to console

---

### 3.3 Run Backtest

```
python main.py backtest
```

Executes a walk-forward backtest on the 20% held-out test set with:
- Position sizing: 2% of balance risked per trade (configurable in `config.py`)
- Random slippage: 0 to 2 XAUUSDc units per entry
- Spread filter: trades skipped if stop-loss distance is less than spread multiplied by 10
- 1:1 risk-reward enforcement

**Output**:
- Performance report printed to console and saved to `results/report.txt`
- Individual trade records saved to `results/trades.csv`

---

### 3.4 View Report

```
python main.py evaluate
```

Re-prints the saved report from `results/report.txt`.

---

### 3.5 View Predictions

```
python main.py predict
```

Prints class probabilities for the last 10 bars:

| Column | Meaning |
|--------|---------|
| DO_NOTHING | Probability the bar should be skipped |
| BUY | Probability of a winning long setup |
| SELL | Probability of a winning short setup |
| HOLD | Probability that no clear edge exists |

---

### 3.6 Full Pipeline

```
python main.py run
```

Runs fetch, train, and backtest sequentially. Requires MT5 to be open and logged in.

---

## 4. Configuration Reference

All parameters are defined in `config.py`. Key settings:

| Parameter | Default | Description |
|-----------|---------|-------------|
| SYMBOL | XAUUSDc | Trading instrument |
| TIMEFRAME_MINUTES | 3 | Bar period |
| LOOKBACK_DAYS | 365 | Historical data range |
| SMA_FAST_PERIOD | 14 | Fast SMA lookback |
| SMA_SLOW_PERIOD | 50 | Slow SMA lookback |
| VROC_PERIOD | 14 | Volume Rate of Change lookback |
| ADX_PERIOD | 14 | Average Directional Index lookback |
| MOMENTUM_SI_PERIOD | 10 | Momentum Strength Index lookback |
| VIX_PROXY_PERIOD | 20 | Synthetic VIX rolling window |
| ATR_PERIOD | 14 | ATR for stop-loss calculation |
| ATR_SL_MULTIPLIER | 1.5 | Stop-loss = ATR * multiplier |
| LABEL_LOOKAHEAD_BARS | 60 | Max bars to wait for TP/SL resolution |
| RR_RATIO | 1.0 | Risk-reward ratio |
| SPREAD_FILTER_MULTIPLIER | 10 | Min SL distance = spread * this value |
| SLIPPAGE_MIN | 0.0 | Minimum random slippage (units) |
| SLIPPAGE_MAX | 2.0 | Maximum random slippage (units) |
| DEFAULT_BET_PCT | 0.02 | Fraction of balance risked per trade |
| STARTING_BALANCE | 10000.0 | Initial account balance (USD) |
| TRAIN_SPLIT_RATIO | 0.80 | Chronological train/test split |

---

## 5. Output Files

| Path | Content |
|------|---------|
| `data/raw_xauusdc_m3.csv` | Raw OHLCV data from MetaTrader 5 |
| `data/featured_data.csv` | Data with all computed features and labels |
| `models/lgbm_model.pkl` | Serialized LightGBM model |
| `results/report.txt` | Backtest performance report |
| `results/trades.csv` | Individual trade records with entry/exit prices, PnL, timestamps |

---

## 6. Metrics Explained

| Metric | Definition |
|--------|------------|
| Win Rate | Percentage of trades that hit take-profit |
| Avg Win % | Mean profit per winning trade as percentage of account balance |
| Avg Loss % | Mean loss per losing trade as percentage of account balance |
| Sharpe Ratio | Annualized risk-adjusted return (mean return / std of returns) |
| Sortino Ratio | Like Sharpe but only considers downside volatility |
| Max Drawdown | Largest peak-to-trough decline in the equity curve |
| Profit Factor | Total gross profit divided by total gross loss |
| Calmar Ratio | Annualized return divided by maximum drawdown |
| Expectancy | Average PnL per trade in USD |

---

## 7. Troubleshooting

| Issue | Solution |
|-------|----------|
| MT5 initialize failed | Ensure MetaTrader 5 is open and logged in |
| No data returned | Add XAUUSDc to Market Watch in MT5 |
| Model file not found | Run `python main.py train` before backtest |
| Import errors | Activate the virtual environment and run `pip install -r requirements.txt` |