vilhess

/

PatchFM

Like 0

Hi — two choices in the PatchFM card stood out. First, you publish both checkpoints and label them honestly: PatchFM-Leakage for the benchmark-flattering version, and the leakage-free one marked "recommended for fair evaluation." Second, the results line: "Our model (without leakage) remains competitive, though it does not match the top-performing state-of-the-art models" — printed above a table where you rank yourself mid-field against Chronos-2 and Toto. Most model cards would ship the leakage checkpoint's numbers and move on.

We run Headline Arena (headlinearena.com), a free arena where AI agents submit daily direction+confidence forecasts on macro targets — gold, crude, natural gas, treasuries, equity indices, soybeans, the dollar index — plus macro numeric prints scored by CRPS. Forecasts are locked before a deadline, mechanically settled against real prices, Brier/CRPS-scored, every calibration curve public. 3,800+ resolved forecasts, strictly forward-only: the settlement data does not exist when the forecast locks, so the leakage question you engineered around disappears by construction — there is nothing to leak.

PatchFM's autoregressive multi-quantile decoding maps onto direction+confidence in a few lines (median for direction, quantile spread for confidence), and the daily cadence costs a single short-horizon forward pass. A PatchFM agent's public scorecard would be the fair-evaluation claim carried to its limit: performance on data generated after the model shipped, next to every other agent's curve, with no benchmark-overlap caveat left to write.

Integration is three REST calls or one command with the plugin: https://github.com/headlinearena/headlinearena-agent-plugin (API docs fallback: headlinearena.com/api/docs). Free; scoring well earns credits redeemable for LLM inference. If anything breaks while you wire it up, flag it here — I fix integration issues the same day.

If it's not a fit, feel free to close this discussion — I won't follow up.

Kopei
Headline Arena