Prevailing level network for sparse, gappy panels

A small PyTorch network that estimates a smooth prevailing level for every location and day of a panel, for two related quantities at once, when the observations are noisy daily means with long gaps and some series missing entirely. It was developed for the paper PM2.5, PM10 ~ Station + Time, Analysis on alphaXiv and powers the sparse panel demo Space.

This repo holds the code and the training recipe, not trained weights. The network learns an embedding for each location and a time basis over a fixed date range, so a fitted model only applies to the panel it was fitted on. To use it, fit it to your own panel: on the 12-station, four-year Beijing data a fit takes a few seconds on a CPU.

Files

model.py: the encoding, the network, the loss and fit / predict
example.py: fits the network to a small synthetic panel and predicts every cell (runs as is)

Model

Each row is one location and day, with up to two targets $y_{1}, y_{2}$ (log values, standardised).

$\begin{aligned} x &= \big[\, e_{\text{area}} + e_{\text{station}},\ b(t) \,\big] \\ h &= \operatorname{SiLU}\!\big(W_2 \operatorname{SiLU}(W_1 x)\big) \\ \mu_1 &= w_1^\top h, \quad \mu_2 = w_2^\top h \end{aligned}$

Location: an 8-dim area embedding plus a station offset that starts near zero and is penalised, so a station with little data falls back to its area.
Time: fixed Gaussian bumps $b_k(t) = \exp\big(-\tfrac12((t - c_k)/w)^2\big)$ with centres every $w$ days, so the level is a smooth function of time.
Shared trunk, two heads: two layers of 64 with SiLU, then one linear head per target. A day where only one target was observed still trains everything the other depends on.

Loss. An observed value is a mean of $n$ readings, so its variance around the level is $\sigma_j^2 / n + \tau_j^2$ , with $\sigma_j$ , $\tau_j$ learned per target. The Gaussian negative log-likelihood is averaged over observed cells only: a missing target is masked, not dropped. Added to it:

a curvature penalty $\lambda \cdot \overline{(\mu_{t+1} - 2\mu_t + \mu_{t-1})^2}$ over every location's predicted series on the whole day grid, observed or not, which keeps the level smooth through gaps;
a small L2 penalty on the station offsets.

Training recipe

Full-batch Adam with a cosine learning-rate schedule. The settings below were chosen by look-ahead cross-validation in the paper:

Setting	Value
curvature penalty $\lambda$ (`rough`)	1e5
bump spacing $w$ (`spacing`)	30 days
hidden width	64
learning rate	5e-3
weight decay	1e-4
station offset L2 (`station_l2`)	1e-3
schedule length	1000
training steps	100
seed	0

For a single-target model, pass use=["PM2.5"] (or the other target) to fit.

Input format

Long format, one row per location × date × target that has data:

column	meaning
`station`	location id
`area`	group the location belongs to (it falls back to this when its own data is thin)
`date`	daily timestamp
`pollutant`	target name, one of `model.TARGETS`
`value`	the daily mean (> 0; modelled on the log scale)
`n`	number of readings behind that mean (weights the observation)

The two target names are set in model.TARGETS (["PM2.5", "PM10"]); change them for other quantities.

from model import Encoding, fit, predict

enc = Encoding.fit(panel, end=panel["date"].max())
model = fit(enc, enc.wide(panel), CFG, steps=100)
level = predict(model, enc, enc.grid())   # every location x day, observed or not

Limitations

Not transferable. Location embeddings and the time basis are specific to one panel and date range. Refit for new locations or dates.
Smoothness is a choice. Cross-validation that scores against noisy readings rewards following short episodes, so it tends to under-smooth; the paper discusses this. Treat rough and spacing as decisions about how smooth the level should be.
Log-scale bias. The model is fitted to logs of means over few readings, so its level sits slightly below a moving average of full data (Jensen's inequality).
A GAM is a strong alternative. In the paper, a mixed-effects GAM with hand-set smoothness does as well or better.
Memory. Training is full batch and the penalty covers every location × day, so memory grows with panel size.

Data used in the paper

Beijing Multi-Site Air Quality, Song Chen (2017), UCI Machine Learning Repository, doi:10.24432/C5RK5G, CC BY 4.0. The demo's sparse readings and predictions are in the dataset prteek/sparse-panel-demo.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

prteek
/

sparse-panel-demo