RESEARCH PREVIEW · BCF

Beacon Forecast

A demand forecaster that watches its own forecast errors day by day, and automatically retrains itself the moment a supply shock or other disruption breaks its assumptions — without waiting for a human to notice.

Launch live demo Read the code
HEADLINE RESULT
A synthetic supply-shock event triggered an automatic retrain within 4 days, with the largest drift z-score (−6.55) of any retrain event across a 2-year, 4-SKU walk-forward simulation.
4 days
from injected shock to automated retrain trigger
−6.55
z-score at the shock-triggered retrain, the largest of any event
4 SKUs × 2yr
walked forward day by day in the simulation
~20
forecast MAE (units/day) after tuning, down from 117+ before a real bug fix
METHOD

Forecast, monitor, retrain — automatically

The drift monitor doesn't wait for a scheduled retrain: it watches forecast error day by day and reacts as soon as the error pattern stops looking normal.

Forecast
A trend + weekly + yearly-seasonality model (Ridge regression, regularized to stay stable on short retrain windows) predicts one day ahead at a time.
Monitor
Every 5 days, a z-test compares the last 14 days of forecast error against the baseline error distribution from right after the last training.
Detect
A Kolmogorov-Smirnov test runs alongside as a secondary check on error *shape*, not just mean — catching variance changes a pure mean-shift test would miss.
Retrain
If |z| exceeds 4.0, the model is automatically refit on the most recent 120 days — a stand-in for an Airflow-triggered Lambda retrain.
Repeat
The cycle continues for the full 2-year simulation, with every retrain event logged and its trigger statistics recorded.
FINDINGS

What the synthetic test runs showed

An early version of the forecaster used plain OLS regression and produced forecasts that exploded to 1,000+ units/day on a series that never exceeds 300 — caused by trend and yearly-seasonality terms becoming collinear on short retrain windows. Switching to Ridge regularization and rescaling the trend feature fixed it, cutting MAE from 117+ to ~20.
The injected supply-shock SKU triggered a retrain within 4 days of the disruption, with a z-score nearly 40% larger in magnitude than any of its own routine periodic retrains.
Non-shocked SKUs still retrain periodically (roughly every 6-10 weeks) as ordinary forecast staleness accumulates — a real, expected MLOps pattern rather than noise, since even a well-fit lightweight model drifts stale over time without an external shock.