Push model using huggingface_hub.

README.md

@@ -1,251 +1,10 @@
 ---
 tags:
-- forecasting
-- time-series
-- zero-shot
-- foundation-models
-datasets:
-- Salesforce/GiftEval
-- autogluon/chronos_datasets
-- Datadog/BOOM
-- Salesforce/GiftEvalPretrain
+- model_hub_mixin
+- pytorch_model_hub_mixin
 ---
 
-Code : https://github.com/vilhess/PatchFM
-
-# A Foundation Model for Univariate Time Series Forecasting
-
-A concise, reproducible recipe for training a transformer-based, patch-to-patch forecasting model for univariate time series. The approach mirrors Large Language Model (LLM) practices (next-token → next-patch) while remaining lightweight compared to a classic LLM and practical.
-
-Our model (with leakage) is deployed on the [TS-Arena benchmark](https://ts-arena.live) and achieves competitive performance against state-of-the-art methods under the name `LITIS/PatchFM-Large`.
-
-## Results on the FEV benchmark:
-
-**Our model (without leakage) is competitive with the state-of-the-art notably compared to [MOIRAI2.0](https://arxiv.org/pdf/2511.11698)**
-
-| Model Name           | Win Rate | Skill Score |
-|----------------------|----------|-------------|
-| (🥇) Chronos-2           | 0.9008   | 0.4728      |
-| (🥈) TiRex               | 0.8092   | 0.4268      |
-| (🥉) TimesFM-2.5         | 0.8085   | 0.4668      |
-| (4) Toto-1.0            | 0.7069   | 0.4110      |
-| (5) **PatchFM (us)**             | 0.6684   | 0.3984      |
-| (6) Moirai-2.0          | 0.6546   | 0.4026      |
-| (7) Chronos-Bolt        | 0.6277   | 0.3889      |
-| (8) Sundial-Base        | 0.4446   | 0.3387      |
-| (9) CatBoost (Recursive)| 0.3362   | 0.2301      |
-| (10) LightGBM (Recursive)| 0.3123   | 0.2168      |
-| (11) AutoTheta           | 0.2938   | 0.0546      |
-| (12) Seasonal Naive      | 0.2058   | 0.0000      |
-| (13) Naive               | 0.1404   | -0.4540     |
-| (14) Drift               | 0.0915   | -0.4578     |
-
-
-## Highlights
-- Next-patch prediction objective (autoregressive, causal)
-- Patch-based representation of time series (tokens ↔ patches)
-- Causal masking self-attention with RoPE (relative positions)
-- Causal RevIN with $\sinh^{-1}$ transform for normalization 
-- SwiGLU feed-forward networks
-- Autoregressive multi-quantile decoding [MOIRAI2.0](https://arxiv.org/pdf/2511.11698)
-- KV-cache for efficient long-horizon inference
-- flips equivariance during inference (optional) [Reverso](https://arxiv.org/pdf/2602.17634v1)
-
-## Quick Start
-
-### from source code
-
-> **⚠️ Important:** Two pretrained versions of PatchFM are available:
-
->
-
-> - **`PatchFM-Leakage`**: trained on the GIFT-Eval datasets. This version may provide higher benchmark performance but includes training data that overlaps with the commonly used evaluation datasets.
-
-> - **`PatchFM`** (**recommended for fair evaluation**): trained without any commonly used benchmark (except BOOM), preventing potential data leakage and ensuring a more reliable assessment of generalization performance.
-
-1. Clone the repository and install dependencies
-```bash
-git clone https://github.com/vilhess/PatchFM
-cd PatchFM
-pip install -r requirements.txt
-```
-2. Run inference with a pretrained model from Huggingface Hub
-
-```python 
-import torch
-
-from configs import PatchFMConfig
-from model import Forecaster
-
-# --- Instantiate model ---
-config = PatchFMConfig(load_from_hub=True, full_leakage=False)  # set full_leakage=True to load the version trained with data leakage
-model = Forecaster(config)
-
-# --- Inference ---
-forecast_horizon = 64
-seq = torch.randn(1, 1024)  # (batch, time)
-pred_median, pred_quantiles = model(seq, forecast_horizon=forecast_horizon, quantiles=[0.1, 0.5, 0.9], flip_equivariance=True)  #  (batch, time), (batch, time, quantiles)
-```
-
-
-### from pip package
-1. Install the package from PyPI
-```bash
-pip install patchfm
-```
-2. Run inference with a pretrained model from Huggingface Hub
-
-```python 
-import torch
-
-from patchfm import Forecaster, PatchFMConfig
-
-config = PatchFMConfig(full_leakage=False)
-# same as above
-pred_median, pred_quantiles = model(seq, forecast_horizon=forecast_horizon, quantiles=[0.1, 0.5, 0.9], flip_equivariance=True)  #  (batch, time), (batch, time, quantiles)
-```
-
-We provide an extended quick start example in [notebooks/tutorial.ipynb](./notebooks/tutorial.ipynb).
-If you dont have suitable hardware you can run the the extended quick start example example also in Google Colab:
-
-<a target="_blank" href="https://colab.research.google.com/drive/17sdf-7luCkv5TaeLj3Z6kIaTDkwkz3VR?usp=share_link">
-  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open Quick Start In Colab"/> 
-</a>
-
-## Method (TL;DR)
-- Patching: Split a context signal of length $w$ into $P_{num} = w / P_{len}$ patches of length $P_{len}$.
-- Causal RevIN: Normalize input signal and denormalize outputs to the original scale without statistics leakage.
-- Architecture: Input residual MLP → stacked Transformer blocks (MHA + SwiGLU FFN, pre-norm, residual) → $|\mathcal{Q}|$ output heads mapping back to patch space.
-- Positional encoding: Rotary Position Embeddings (RoPE) applied to queries/keys.
-- Training: Multi-quantile (pinball) loss across positions, elements, and quantiles $\mathcal{Q}$.
-- Inference: Predict next patch; roll out autoregressively for long horizons.
-- KV-cache: during inference, cache keys/values to avoid redundant computations.
-- Flip-equivariance: during inference, flip input sequence and average predictions to improve robustness (at cost of doubling batch size).
-
-## Problem Formulation
-Given context patches $x_{p_1}, \ldots, x_{p_n}$, predict the next patch $x_{p_{i+1}}$ for each position $i$ using only past patches (causality). The model outputs quantiles $\{\hat{x}_{p_{i+1}}^{(q)}: q \in \mathcal{Q}\}$ with median (q=0.5) as the point forecast.
-
-## Loss: Multi-Quantile (Pinball)
-For residual $u = x - \hat{x}^{(q)}$:
-$$\rho_q(u) = \begin{cases} q\,u, & u \ge 0,\\ (q-1)\,u, & u < 0. \end{cases}$$
-Aggregate over positions, patch elements, and quantiles.
-
-## Architecture
-- Input MLP: $\mathbb{R}^{P_{len}} \to \mathbb{R}^{dim}$ residual 2-layer MLP (ReLU)
-- Multi-Head Attention: causal mask, RoPE; queries/keys/values per head
-- FFN: SwiGLU (SiLU-gated), pre-norm + residual
-- Output heads: |Q| linear maps $\mathbb{R}^{dim} \to \mathbb{R}^{P_{len}}$ (one per quantile)
-
-### Model Details
-- Patch size: 32
-- Max context: 32 patches (1024 steps)
-- Forecast horizon: 32 steps per forward pass
-- Quantiles $\mathcal{Q}$: {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}
-- Layers: 6
-- Attention heads: 32 (head dim 64)
-- Model dim: 2048
-- Parameters: ~300M
-
-## Inference
-- Single step: predict next patch ($P_{len}$ values)
-- Long-horizon: append prediction to context and repeat (optionally drop oldest patch to keep window fixed)
-- Flip-equivariance [Reverso](https://arxiv.org/pdf/2602.17634v1): optionally flip input sequence and average predictions to improve robustness (at cost of doubling batch size):
-$$y = \frac{1}{2} \left( f(x) - f(-x) \right)$$
-
-- ### Autoregressive Inference with Quantile Forecasting ([Moirai 2.0](https://arxiv.org/pdf/2511.11698v1))
-During autoregressive inference, the model generates forecasted values patch by patch. At each time step, the predicted patch is fed back into the model as input for the next step. This iterative process continues until the desired forecast horizon is reached.
-
-When performing quantile forecasting, the situation becomes more complex. Instead of producing a single patch per step, the model outputs multiple patches corresponding to different quantiles (e.g., 0.1, 0.5, 0.9). Since the model expects a single patch for the next time step, it is not straightforward to feed all quantile predictions back into the model simultaneously.
-
-A common workaround is to feed only the median prediction (the 0.5 quantile) back into the model at each step. While this approach preserves the autoregressive structure, it discards the uncertainty information captured by the other quantiles.
-
-An alternative approach is **autoregressive multi-quantile decoding**, as proposed in [Moirai 2.0](https://arxiv.org/pdf/2511.11698v1). This method enables consistent autoregressive generation while preserving the full predictive distribution across quantiles. However, it is computationally more expensive than the median-only approach as it requires duplicating the context for each quantile.
-
-<div style="display: flex; gap: 10px; align-items: flex-start;">
-  <div>
-    <img src="assets/classic_forecast_animation.gif" alt="Classic Autoregressive Inference" width="450">
-    <p style="text-align:center;">Classic Autoregressive Inference</p>
-  </div>
-  <div>
-    <img src="assets/quantile_forecast_animation.gif" alt="Autoregressive Multi-Quantile Decoding" width="450">
-    <p style="text-align:center;">Autoregressive Multi-Quantile Decoding</p>
-  </div>
-</div>
-
-The algorithm proceeds as follows:
-
-1. **Initialization**  
-   Start with the initial context window of observed data  
-   **Shape:** `(BS × L)`  
-   - `BS`: batch size  
-   - `L`: context length  
-   - `P`: patch size  
-   - `Q`: number of quantiles  
-   - `H`: forecast horizon  
-   - `i=1`: current algorithm step
-
-2. **First Quantile Prediction (Forward Pass)**  
-   Predict the quantiles for the next patch using the current context.  
-   **Output shape:** `(BS × P × Q)`
-
-3. **Context Duplication**  
-   For each predicted quantile, create a separate context by appending the corresponding predicted patch to the current context.  
-   This increases the number of contexts by a factor of `Q` at each step.  
-   **New context shape:** `(BS × Q × i(L + P))`
-
-4. **Next Forward Pass**  
-   For each duplicated context, predict the quantiles of the next patch.  
-   **Output shape:** `(BS × Q × P × Q)`
-
-5. **Quantile Collapse**  
-   - Permute and reshape the predictions to aggregate all possible quantile paths:  
-     **Intermediate shape:** `(BS × P × Q²)`  
-   - Compute the quantiles across the `Q²` predictions to obtain the final quantile estimates for the next patch.  
-     **Final shape:** `(BS × P × Q)`
-   - Increment the step counter `i ← i + 1`.
-
-6. **Iteration**  
-   Repeat Steps 3–5 until the forecast horizon `H` is reached, i.e., until the total number of predicted time steps satisfies  
-   `i × P ≥ H`.
-
-This procedure preserves predictive uncertainty across quantiles while maintaining the autoregressive structure of the model. Although it is computationally more expensive than feeding only the median prediction (0.5 quantile) back into the model, it remains tractable in practice and enables consistent multi-quantile forecasting.
-
-⚠️ **Warning**  
-With this strategy, the median prediction (0.5 quantile) does **not necessarily** match the prediction obtained by autoregressively feeding only the median patch back into the model at each step.
-
-This discrepancy arises because the *quantile collapse* step aggregates predictions across all possible quantile paths. As a result, the median is computed from the combined multi-path distribution rather than from a single deterministic trajectory, which can lead to different estimates compared to the single-path (median-only) autoregressive approach.
-
-## Datasets
-- GIFT-Eval pretraining dataset [GIFT]: aligned with the GIFT-Eval dataset but without data leakage issue with the benchmark. The dataset contains approximately 71 univariate and 17 multivariate time series datasets from various
-domains and various frequencies. After preprocessing, this yields approximately 600K univariate series. 
-- Chronos synthetic datasets [Chronos]: two large synthetic datasets generated with Chronos, one with TSMixup and one with KernelSynth. Each contains approximately 10 million univariate series and 1 million respectively, each signal of length 1024.
-- Artificial: ~1M synthetic series (sinusoidal, linear, polynomial, logarithmic) plus mixtures via TSMixup [Chronos]; Gaussian Process samples via KernelSynth (mixtures of RBF/periodic/linear kernels with swept hyperparameters).
-- BOOM: Approximately 5 million time series of length 1024 sampled from the BOOM dataset [BOOM]. It is important to note that the BOOM dataset, which is also used as an evaluation benchmark, is included in the training data of both PatchFM variants (with and without benchmark leakage) to increase data diversity. Consequently, neither version can be fairly evaluated on the BOOM benchmark. Upon request, we can provide an alternative checkpoint trained without any BOOM data, enabling a fair evaluation on this benchmark.
-
-## Repository Layout
-
-- `model/training/` — main PatchFM model class
-
-  - `modules.py` - core modules (Residual Layers, MHA, SwiGLU, RoPE, Transformer Encoder, ...)
-  - `revin.py` — causal RevIN
-  - `loss.py` — multi-quantile (pinball) loss
-  - `trainer.py` — PyTorch Lightning trainer class
-
-- `model/inference/` — main PatchFM model class for inference 
-  - `modules.py` — core modules with caching support
-  - `forecaster.py` — Forecasting model and rollout logic
-
-- `dataset/` — data loading and preprocessing
-  - `artificial.py` — synthetic dataset : artificial signals + TSMixup + KernelSynth
-  - `gift.py` — GIFT-Eval pretraining dataset loading and preprocessing
-  - `get_data.py` — utility to fetch and preprocess datasets
-  - `chronosdata.py` — loading of the synthetic datasets generated with Chronos (TSMixup and KernelSynth) with download functions integrated
-  - `boom.py` — loading of the BOOM dataset with download functions integrated
-
-- `configs/` — model and training configurations
-- `notebooks/inference` — how to load a trained model and generate forecasts
-- `training.py` — training script using PyTorch Lightning
-
-## Acknowledgements
-We thank the authors of the following repositories for inspiration and code snippets:
-- [TiRex](https://github.com/NX-AI/tirex)
+This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
+- Code: [More Information Needed]
+- Paper: [More Information Needed]
+- Docs: [More Information Needed]

config.json

@@ -1,5 +1,5 @@
 {
-  "ckpt_path": "ckpts/pretrain_boom.ckpt",
+  "ckpt_path": "ckpts/toto.ckpt",
   "compile": true,
   "d_model": 2048,
   "load_from_hub": false,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cf4663d0747faed47a686eaeab63ea87c96fcb1db4e90ae4ce2d290d77a86ced
-size 1275010080
+oid sha256:0ef55a59121cebf8188f538dc970e668b56881e61321fe0bd297529651b3442b
+size 1380957032