README.md

metadata

title: FutureQuery forecast-service
sdk: docker
app_port: 8008

FutureQuery forecast-service

An ensemble time-series forecasting service for prediction-market probabilities. It blends Google TimesFM 2.x and Amazon Chronos-2, applies superforecasting-style calibration, tracks its own Brier score over time, and exposes a small FastAPI surface plus a LangGraph research pipeline.

Replaces the old single-model timesfm-service/.

Quick start

cd forecast-service
pip install -r requirements.txt
python main.py        # starts on http://localhost:8008

The first run downloads model weights from HuggingFace (~2GB — this is normal and only happens once). If a model fails to load (offline, no GPU, OOM) the service still starts and falls back to a naive forecast, so the API never hard-fails.

API

POST /forecast

// request
{
  "prices": [0.48, 0.50, 0.51, 0.52, ...],  // historical yes_prices
  "covariates": [[...], ...],                // optional related series
  "question_type": "event",                  // "event" | "numeric"
  "horizon": 5,
  "question": "Will X happen by 2027?"       // optional, stored for calibration
}

// response
{
  "timesfm":  { "forecast": [...], "trend": "up" },
  "chronos2": { "forecast": [...], "trend": "up" },
  "ensemble": { "forecast": [...], "trend": "up", "brier_estimate": 0.21 },
  "use_forecast": true,        // false if event w/ no price history, or series < 10 pts
  "warning": null,             // e.g. "Series too short for reliable forecast"
  "id": "…"                    // calibration row id
}

Ensemble logic (ensemble.py)

Step	Rule
Length guard	< 10 points → use_forecast=false + warning
Event blend	0.5·TimesFM + 0.5·Chronos-2 (equal weight — both models show negative R² on binary event markets, so neither dominates)
Numeric blend	0.4·TimesFM + 0.6·Chronos-2 (Chronos-2 wins on fev-bench)
Base-rate prior	if `
Extremise	only if confidence > 0.70: 0.5 + 1.3·(p − 0.5)

GET /calibration

Returns the realised Brier score, a reliability curve (decile bins of predicted vs. observed frequency), and directional accuracy over all resolved forecasts — so you can benchmark against superforecasters (~0.15 Brier) over time. Data is stored in db.sqlite.

POST /resolve

{ "id": "<forecast id>", "outcome": 1.0 }   // 1.0 = YES happened, 0.0 = NO

Records a market's actual outcome so it feeds the calibration scoring.

GET /health

Liveness + which models actually loaded.

LangGraph pipeline (Phase 3)

pipeline.py can run the full SCAN → TRIAGE → SYNTHESISE loop across Polymarket, Kalshi, Metaculus, and Manifold for local CLI experiments. The browser Pipeline does not trust that raw scan as its market source: the Next.js proxy sends a classified market_context built by the Predict Trust Layer, and the Python graph uses only exact/likely classified markets for consensus.

python pipeline.py "Will UK interest rates fall below 4% by Jan 2027?"

At the checkpoint, press Enter to accept or type override 0.42 to set your own probability.

Files

File	Purpose
main.py	FastAPI app, startup model loading, endpoints
models.py	TimesFM / Chronos-2 wrappers with graceful fallback
ensemble.py	Blending, base-rate prior, extremisation (numpy-only, testable)
calibration.py	SQLite store + Brier / reliability / directional scoring
pipeline.py	LangGraph SCAN→TRIAGE→SYNTHESISE workflow
requirements.txt	Python dependencies

Notes

• FORECAST_PORT overrides the default port (8008).
• The Next.js app calls /forecast only when FORECAST_URL is set and an exact/likely Polymarket market with a CLOB token was selected.
• The Next.js caller defaults to FORECAST_TIMEOUT_MS=90000 and CALIBRATION_TIMEOUT_MS=10000 (server-only env overrides, capped at 240 seconds) so model quality and cold starts are not cut short by the old short timeout.
• Returned ensembles are supporting Momentum signals only; they do not blend into the headline probability, and the app degrades gracefully when the service is offline.

Browser pipeline (no terminal)

The same SCAN → TRIAGE → ENSEMBLE → SYNTHESISE → CHECKPOINT loop is exposed over HTTP so it can be driven entirely from the web app:

• POST /pipeline/start — { "question": "...", "question_type": "event", "market_context": { ... } } runs to the human checkpoint and returns the full state plus a thread_id. market_context is supplied by the Next.js proxy; with it present, related/rejected markets are provenance only and cannot drive the Pipeline probability or price-history selection.
• POST /pipeline/resume — { "thread_id": "...", "resume": "" } to accept, or { "resume": "override 0.42" } to override.

In the Next.js app open /pipeline (e.g. http://localhost:3000/pipeline): type a question, watch the scan / triage / arbitrage / ensemble panels populate, then Approve or Override the synthesised probability. Requires FORECAST_URL to be set so the app can reach this service.