# M5 Forecasting: Statistical Models Comparison

## Performance Summary

| Model          | RMSE      | MAE       | MAPE    | Model Size | Repo |
|----------------|-----------|-----------|---------|------------|------|
| **SARIMAX**    | 2759.70 | 2260.25 | 4.98% | 85.30 MB | [rishini/NPN-sarimax](https://huggingface.co/rishini/NPN-sarimax) |
| **Prophet**    | 4860.67 | 4038.73 | 8.73% | 0.18 MB | [rishini/NPN-prophet](https://huggingface.co/rishini/NPN-prophet) |
| **ARIMA**      | 6459.70 | 4852.62 | 10.48% | 6.56 MB | [rishini/NPN-arima](https://huggingface.co/rishini/NPN-arima) |
| **LightGBM** (per-series) | N/A (WRMSSE=145.56) | | | 106.1 MB | [rishini/NPN](https://huggingface.co/rishini/NPN) |

## Key Findings

### 1. SARIMAX Wins (Best Accuracy)
- **Lowest RMSE**: 2,759.70 (4.98% MAPE)
- **Best fit**: SARIMAX(2,1,1)(1,1,1,7) captures weekly seasonality
- **Exogenous boost**: SNAP indicators and event dummies improve predictions
- **Trade-off**: Largest model file (85MB) due to complex state space representation

### 2. Prophet (Best Interpretability)
- **RMSE**: 4,860.67 (8.73% MAPE)
- **Strengths**: Fast training, automatic seasonality detection, built-in uncertainty intervals
- **Weaknesses**: Underperforms on aggregate-level predictions
- **Trade-off**: Smallest model (180KB), fastest to deploy

### 3. ARIMA (Baseline Simplicity)
- **RMSE**: 6,459.70 (10.48% MAPE)
- **Best config**: ARIMA(3,1,1) with deterministic trend
- **Strengths**: Simple, interpretable, smallest non-Prophet model
- **Weaknesses**: No seasonality, no exogenous variables, poorest fit

### 4. LightGBM (Per-Series Champion)
- **WRMSSE**: 145.56 (beats naive baselines by 55-69%)
- **Advantage**: Predicts all 30,490 series individually
- **Trade-off**: Not directly comparable (different granularity)

## Why SARIMAX Outperforms?

1. **Weekly Seasonality**: Retail demand has strong 7-day cycles (weekends higher)
2. **Exogenous Signals**: SNAP eligibility and events directly impact demand
3. **Autocorrelation**: Captures persistence in sales patterns
4. **Differencing**: (d=1) removes trend, focusing on changes

## Why These Models Don't Beat LightGBM?

| Aspect | LightGBM | Statistical Models |
|--------|----------|-------------------|
| Granularity | 30,490 individual series | 1 aggregate series |
| Features | 34 engineered features | 6-12 basic features |
| Flexibility | Non-linear relationships | Linear/AR/MA assumptions |
| Cross-series learning | Store/dept/item interactions | No sharing across series |
| Deployment | 40 per-store models | 3 aggregate models |

The statistical models operate on **aggregate** sales (total ~34,000/day), while LightGBM models each of the **30,490 individual series** with specialized features. The statistical approach serves as a solid baseline but cannot match the per-series precision of gradient boosting.

## Model Selection Guide

Use **SARIMAX** when:
- You need the best statistical baseline
- Exogenous variables (events, promotions) are important
- Weekly seasonality dominates

Use **Prophet** when:
- Fast experimentation is needed
- Interpretability is key
- Multiple seasonalities exist

Use **ARIMA** when:
- Simple baseline is sufficient
- No strong seasonality
- Minimal computational budget

Use **LightGBM** when:
- Maximum accuracy is required
- Per-series predictions needed
- GPU is available