m0hamadk/paranal-seeing-xgboost

Paranal Seeing Prediction - XGBoost

Model Description

An XGBoost regression model that predicts real astronomical seeing (a measure of atmospheric turbulence that limits telescope image quality) at ESO's Paranal Observatory, trained on real, site-specific instrument data (2016-2026) rather than generic weather-forecast features. Hyperparameters were tuned with a 9-parameter Optuna search, validated on a held-out block of weeks distinct from both training and final test data.

The model predicts log(1 + seeing); invert with np.expm1(prediction) to recover seeing in arcsec.

An ensemble of this model with a companion neural network was also tested and improved R\u00b2 by only 0.0025 (within noise) - not enough to justify a two-model production system, so this XGBoost model alone is the recommended model.

Intended Uses & Limitations

Intended use: retrospective / statistical analysis of the relationship between real atmospheric conditions and astronomical seeing at Paranal (e.g. site-characterization research, as a reference/starting point for building an equivalent model at another observatory with comparable instrumentation).

Not intended for: live, forward-looking seeing forecasting. Several of the strongest input features (local meteo-tower and radiometer readings) are hyper-local to Paranal's own instrumentation and are not produced by any general weather-forecast model; Paranal's own ambient database itself updates once per local night rather than in real time. A deployable forecasting tool would need to be re-trained on forecast-compatible inputs (e.g. reanalysis pressure-level data), trading some accuracy for lead-time forecastability.

Out-of-distribution use: this model is specific to Paranal's climate and altitude (2635 m, Atacama desert). Applying it to another site's conditions without retraining is not recommended - see feature_ranges.json (included) for the valid input range seen during training; inputs far outside these ranges should not be trusted.

Training Data

Trained on the companion dataset: ESO Paranal Real Seeing + Site Instrument Dataset (2016-2026), 36,706 real hourly observations. See the dataset card for full column descriptions and provenance.

Training Procedure

• Feature engineering: circular sine/cosine encoding of wind direction; 3-hour rolling means and 3-hour trends (current minus 3-hours-ago) for temperature, wind speed, vertical-wind turbulence, and pressure; near- surface temperature gradient; wind shear between 10 m and 30 m.
• Target transform: log(1 + seeing), to correct a heavy right-skew (~92% of observations fall between 0.5" and 1.5"; under 0.3% exceed 3").
• Split: chronological, block-based (weekly blocks; whole blocks assigned to train/validation/test, never split within a block) to avoid leakage from the strong hour-to-hour autocorrelation of this data. The most recent 15% of weeks were held out as a genuinely unseen future test set.
• Hyperparameter search: Optuna, 80 trials, objective = validation-set MAE (log scale). Tuned parameters: n_estimators, learning_rate, max_depth, subsample, colsample_bytree, min_child_weight, reg_alpha, reg_lambda, gamma. Early stopping (50 rounds) against the validation set.

Evaluation Results

Metric	Value	Notes
R\u00b2	0.696	Chronological hold-out, most recent 15% of weeks
MAE	0.160 arcsec	Real units, after inverse log-transform
RMSE	0.225 arcsec

For comparison, a deep neural network trained on the same data/split reached R\u00b2 = 0.674; linear/ridge/polynomial regression and random forest baselines all scored substantially lower (R\u00b2 < 0.10 and ~0.45 respectively), consistent with seeing depending non-linearly on these inputs.

See feature_importance.csv (included) for the full permutation-importance ranking. The top features are wind direction (sine component), 2 m temperature, and 30 m wind speed; a 3-hour rolling mean of 10 m wind speed ranks 4th, confirming recent atmospheric trend carries genuine signal beyond the instantaneous state.

How to Use

import numpy as np
import pandas as pd
from xgboost import XGBRegressor

FEATURE_ORDER = [
    "airmass", "pressure",
    "temp_2m", "temp_30m", "temp_ground",
    "temp_gradient_ground_30m",
    "dew_point_depression_2m",
    "wind_speed_10m", "wind_speed_30m",
    "wind_shear_10_30",
    "wind_dir_sin", "wind_dir_cos",
    "wind_w_std",
    "rain_intensity",
    "ir_temperature", "pwv",
    "temp_2m_roll3h_mean", "temp_2m_trend_3h",
    "wind_speed_10m_roll3h_mean", "wind_speed_10m_trend_3h",
    "wind_speed_30m_roll3h_mean", "wind_speed_30m_trend_3h",
    "pressure_trend_3h", "wind_w_std_roll3h_mean",
]

model = XGBRegressor()
model.load_model("xgb_model.json")

# `recent_hours` must be a DataFrame of >= 4 consecutive real hourly
# observations (oldest first) with the raw columns described in the dataset
# card, because the rolling/trend features need 3 hours of history - a
# single isolated snapshot is not sufficient. See inference_example.py for a
# full worked example including feature reconstruction.
prediction_log = model.predict(recent_hours[FEATURE_ORDER].iloc[[-1]])
seeing_arcsec = np.expm1(prediction_log)[0]

See inference_example.py (included) for a complete, runnable example that reconstructs all engineered features from raw hourly columns.

Citation / Acknowledgement

Trained on data from the European Southern Observatory's public science archive (http://archive.eso.org). Please credit ESO if you build on this model or dataset.