federicosabbadini commited on
Commit
891e296
Β·
verified Β·
1 Parent(s): 1f59d0d

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +77 -58
README.md CHANGED
@@ -1,102 +1,121 @@
1
  ---
2
- license: apache-2.0
3
  tags:
4
  - time-series
5
  - forecasting
6
  - chronos
7
  - patch-aliasing
8
- language:
9
- - en
10
  ---
11
 
12
- # Chronos-Bolt Patch Sweep
13
 
14
- A collection of **Chronos-Bolt Tiny** models retrained **from scratch** (random weights) on the official [Chronos pre-training corpus](https://huggingface.co/datasets/autogluon/chronos_datasets) (TSMixup 10M + KernelSynth 1M), varying only the **patch geometry** (input_patch_size `P` and input_patch_stride `S`).
15
 
16
- The goal is to study **structural aliasing** in patched time-series transformers: how the choice of patch size and stride affects the model's ability to represent different frequency components.
17
 
18
- ## Models
19
 
20
- | Subfolder | P | S | Overlap Ratio | Approx Patches | Params | Final Loss | Speed (it/s) |
21
- |-----------|---|---|---------------|-----------------|--------|------------|--------------|
22
- | `p16-s16-seed42` | 16 | 16 | 0.00 (contiguous) | 128 | 8.65M | 36.09 | 4.94 |
23
- | `p16-s12-seed42` | 16 | 12 | 0.25 | 170 | 8.65M | 35.34 | 6.78 |
24
- | `p16-s8-seed42` | 16 | 8 | 0.50 | 255 | 8.65M | 36.97 | 3.80 |
25
- | `p16-s4-seed42` | 16 | 4 | 0.75 | 509 | 8.65M | 36.70 | 3.58 |
26
- | `p8-s8-seed42` | 8 | 8 | 0.00 (contiguous) | 256 | 8.63M | 37.62 | 7.03 |
27
- | `p24-s24-seed42` | 24 | 24 | 0.00 (contiguous) | 86 | 8.67M | 34.90 | 4.50 |
28
 
29
- ### Experimental design
30
 
31
- - **Overlap axis** (fixed P=16, varying S): `S=16` (no overlap) β†’ `S=12` (25%) β†’ `S=8` (50%) β†’ `S=4` (75%). Tests whether overlapping patches reduce aliasing artifacts.
32
- - **Patch-size axis** (contiguous, S=P): `P=8` β†’ `P=16` β†’ `P=24`. Tests how the patch size itself (the effective sampling window) affects frequency representation.
 
 
 
 
 
 
33
 
34
- All other hyperparameters are fixed at the official Chronos-Bolt Tiny values so any downstream difference is attributable to P/S alone.
35
 
36
- ## Training details
37
 
38
- - **Base architecture**: [amazon/chronos-bolt-tiny](https://huggingface.co/amazon/chronos-bolt-tiny) (T5-based, ~8.65M params)
39
- - **Weights**: trained from scratch (random init), NOT fine-tuned from the official checkpoint
40
- - **Dataset**: `autogluon/chronos_datasets` β€” TSMixup 10M + KernelSynth 1M (official Chronos corpus)
41
- - **Steps**: 10,000 (vs official 200k β€” reduced compute budget; all models compared only against each other)
42
- - **Batch size**: 32
43
- - **Optimizer**: AdamW (fused), LR=1e-3, linear decay, no warmup
44
- - **Precision**: fp32 + TF32 matmuls on NVIDIA RTX 5060 Laptop GPU
45
- - **Context length**: 2048 tokens
46
- - **Prediction length**: 64 steps
47
- - **Quantiles**: 9 (0.1 to 0.9)
48
 
49
- ## Usage
 
 
50
 
51
  ```python
52
  from chronos import ChronosBoltPipeline
53
  import torch
54
 
55
- # Load any single model
56
- pipeline = ChronosBoltPipeline.from_pretrained(
57
  "federicosabbadini/chronos-bolt-patch-sweep",
58
- subfolder="p16-s16-seed42",
59
  device_map="cuda",
60
  torch_dtype=torch.float32,
61
  )
62
 
 
 
 
 
 
 
63
  # Forecast
64
- context = torch.tensor([...]) # your time series as a 1D tensor
65
- forecast = pipeline.predict(context, prediction_length=64)
66
- # forecast shape: (1, num_samples, 64)
 
67
  ```
68
 
69
- ### Load all models for comparison
70
 
71
  ```python
72
- from chronos import ChronosBoltPipeline
73
- import torch
74
-
75
  REPO = "federicosabbadini/chronos-bolt-patch-sweep"
76
- MODEL_NAMES = [
77
- "p16-s16-seed42", "p16-s12-seed42", "p16-s8-seed42",
78
- "p16-s4-seed42", "p8-s8-seed42", "p24-s24-seed42",
79
- ]
80
 
81
  models = {}
82
- for name in MODEL_NAMES:
83
  models[name] = ChronosBoltPipeline.from_pretrained(
84
- REPO, subfolder=name,
85
- device_map="cuda", torch_dtype=torch.float32,
86
  )
 
 
 
 
 
87
  ```
88
 
89
- ## Files per model
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
- Each subfolder contains:
92
- - `config.json` β€” HuggingFace model config (includes `chronos_config` with P, S values)
93
- - `model.safetensors` β€” trained weights (~33 MB)
94
- - `run_config.json` β€” full provenance + training result metadata
95
- - `loss_history.npy` β€” per-step training loss array (10,000 values)
96
- - `loss_curve.png` β€” training loss plot
97
 
98
- The root `manifest.csv` summarizes all finished runs.
99
 
100
- ## Citation
101
 
102
- This work is part of a research project on patch aliasing in time-series transformers. The base model architecture and training pipeline are adapted from [Chronos](https://github.com/amazon-science/chronos-forecasting) (Apache-2.0).
 
1
  ---
2
+ language: en
3
  tags:
4
  - time-series
5
  - forecasting
6
  - chronos
7
  - patch-aliasing
8
+ - t5
9
+ license: apache-2.0
10
  ---
11
 
12
+ # Chronos-Bolt Patch Sweep β€” Studio sull'Aliasing Strutturale
13
 
14
+ Collezione di modelli **Chronos-Bolt Tiny** (~8.65M parametri) riallenati da zero con diverse geometrie di patch (P, S) per studiare l'aliasing strutturale introdotto dall'embedding a patch nei transformer per serie temporali.
15
 
16
+ ## Risultati principali
17
 
18
+ L'embedding a patch proietta P campioni consecutivi in un singolo token. Quando un'oscillazione completa esattamente un intero numero di cicli nella finestra di P campioni (`cpp = freq Γ— P / fs = intero`), la proiezione lineare cancella l'oscillazione. Questo produce **nulli nella risposta in frequenza** a posizioni prevedibili: `f_null = k Γ— fs / P`.
19
 
20
+ | Ipotesi | Esito |
21
+ |---------|-------|
22
+ | La larghezza P della patch determina la posizione dei nulli | **Confermata** β€” nulli a kΓ—fs/P |
23
+ | I nulli cadono a cpp intero indipendentemente da P | **Confermata** β€” P=8, P=16, P=24 collassano tutti a cpp intero |
24
+ | L'overlap (stride ridotto) mitiga il nullo | **Rigettata** β€” S=4 collassa quanto S=16 |
25
+ | La banda utile del forecaster Γ¨ fs/P | **Confermata** β€” recovery crolla sopra questa soglia |
 
 
26
 
27
+ ## Modelli
28
 
29
+ | Cartella | P | S | Overlap | Step | Stato | Ruolo |
30
+ |----------|---|---|---------|------|-------|-------|
31
+ | `p16-s12-seed42` | 16 | 12 | 25% | 100k | done | Asse stride |
32
+ | `p16-s8-seed42` | 16 | 8 | 50% | 100k | done | Asse stride |
33
+ | `p16-s4-seed42` | 16 | 4 | 75% | 100k | done | Asse stride |
34
+ | `p8-s8-seed42` | 8 | 8 | 0% | 100k | done | Asse patch-size |
35
+ | `p24-s24-seed42` | 24 | 24 | 0% | 100k | done | Asse patch-size |
36
+ | `p16-s16-seed42` | 16 | 16 | 0% | 10k | **stale** | Sostituito dall'ufficiale |
37
 
38
+ > **Nota su p16-s16.** Il retrain p16-s16-seed42 si è fermato a ~33k step e i pesi su HF sono quelli vecchi a 10k. Per il confronto P=16 S=16 usiamo direttamente il modello ufficiale [`amazon/chronos-bolt-tiny`](https://huggingface.co/amazon/chronos-bolt-tiny) (200k step, corpus completo, stessa geometria P=16 S=16). È il miglior anchor possibile per l'asse stride.
39
 
40
+ ## Training
41
 
42
+ - **Architettura:** T5-based (Chronos-Bolt Tiny), ~8.65M parametri
43
+ - **Dataset:** [autogluon/chronos_datasets](https://huggingface.co/datasets/autogluon/chronos_datasets) β€” TSMixup 10M + KernelSynth 1M, interleaving 9:1
44
+ - **Step:** 100,000 (streaming da HF, shuffle buffer 10k)
45
+ - **Ottimizzatore:** AdamW fused, LR 1e-3 β†’ 0 (linear decay), no warmup
46
+ - **Batch size:** 32, gradient clipping 1.0
47
+ - **Precisione:** fp32 + TF32 (RTX 5060 Laptop, sm_120)
48
+ - **Context/Prediction:** 2048 / 64
 
 
 
49
 
50
+ Ogni modello Γ¨ allenato con lo stesso seed (42), stesso dataset, stessi iperparametri. L'unica variabile Γ¨ la geometria della patch (P, S).
51
+
52
+ ## Come usare
53
 
54
  ```python
55
  from chronos import ChronosBoltPipeline
56
  import torch
57
 
58
+ # Caricare un singolo modello retrained
59
+ pipe = ChronosBoltPipeline.from_pretrained(
60
  "federicosabbadini/chronos-bolt-patch-sweep",
61
+ subfolder="p8-s8-seed42",
62
  device_map="cuda",
63
  torch_dtype=torch.float32,
64
  )
65
 
66
+ # Caricare il modello ufficiale come reference P=16 S=16
67
+ official = ChronosBoltPipeline.from_pretrained(
68
+ "amazon/chronos-bolt-tiny",
69
+ device_map="cuda",
70
+ )
71
+
72
  # Forecast
73
+ import numpy as np
74
+ context = torch.tensor(np.random.randn(512), dtype=torch.float32)
75
+ forecast = pipe.predict(context, prediction_length=64) # [1, 9, 64] β€” 9 quantili
76
+ median = forecast[0, 4, :] # quantile 0.5
77
  ```
78
 
79
+ ### Caricare tutti i modelli per confronto
80
 
81
  ```python
 
 
 
82
  REPO = "federicosabbadini/chronos-bolt-patch-sweep"
83
+ VARIANTS = ["p16-s12-seed42", "p16-s8-seed42", "p16-s4-seed42",
84
+ "p8-s8-seed42", "p24-s24-seed42"]
 
 
85
 
86
  models = {}
87
+ for name in VARIANTS:
88
  models[name] = ChronosBoltPipeline.from_pretrained(
89
+ REPO, subfolder=name, device_map="cuda", torch_dtype=torch.float32
 
90
  )
91
+
92
+ # Aggiungere l'ufficiale come p16-s16
93
+ models["p16-s16 (official)"] = ChronosBoltPipeline.from_pretrained(
94
+ "amazon/chronos-bolt-tiny", device_map="cuda"
95
+ )
96
  ```
97
 
98
+ ## Struttura del repository
99
+
100
+ ```
101
+ β”œβ”€β”€ manifest.csv # tabella riassuntiva di tutti i run
102
+ β”œβ”€β”€ p16-s12-seed42/
103
+ β”‚ β”œβ”€β”€ config.json # config del modello (include P, S)
104
+ β”‚ β”œβ”€β”€ model.safetensors # pesi finali (100k step)
105
+ β”‚ β”œβ”€β”€ run_config.json # iperparametri del training
106
+ β”‚ β”œβ”€β”€ loss_history.npy # curva di loss
107
+ β”‚ └── loss_curve.png # plot della loss
108
+ β”œβ”€β”€ p16-s8-seed42/ # stessa struttura
109
+ β”œβ”€β”€ p16-s4-seed42/
110
+ β”œβ”€β”€ p8-s8-seed42/
111
+ β”œβ”€β”€ p24-s24-seed42/
112
+ └── p16-s16-seed42/ # ⚠️ stale β€” pesi a 10k, non usare
113
+ ```
114
 
115
+ ## Codice sorgente
 
 
 
 
 
116
 
117
+ Il codice di training, valutazione e analisi Γ¨ nel repository GitHub: i script di sweep frequenziale, il notebook inspector, e i generatori di segnali sintetici con injection controllata.
118
 
119
+ ## Licenza
120
 
121
+ Apache 2.0 β€” stesso della famiglia Chronos.