federicosabbadini commited on
Commit
3bdb715
Β·
verified Β·
1 Parent(s): f16d548

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +44 -100
README.md CHANGED
@@ -1,121 +1,65 @@
1
- ---
2
- language: en
3
- tags:
4
- - time-series
5
- - forecasting
6
- - chronos
7
- - patch-aliasing
8
- - t5
9
- license: apache-2.0
10
- ---
11
 
12
- # Chronos-Bolt Patch Sweep β€” Studio sull'Aliasing Strutturale
13
 
14
- Collezione di modelli **Chronos-Bolt Tiny** (~8.65M parametri) riallenati da zero con diverse geometrie di patch (P, S) per studiare l'aliasing strutturale introdotto dall'embedding a patch nei transformer per serie temporali.
15
 
16
- ## Risultati principali
17
 
18
- L'embedding a patch proietta P campioni consecutivi in un singolo token. Quando un'oscillazione completa esattamente un intero numero di cicli nella finestra di P campioni (`cpp = freq Γ— P / fs = intero`), la proiezione lineare cancella l'oscillazione. Questo produce **nulli nella risposta in frequenza** a posizioni prevedibili: `f_null = k Γ— fs / P`.
19
 
20
- | Ipotesi | Esito |
21
- |---------|-------|
22
- | La larghezza P della patch determina la posizione dei nulli | **Confermata** β€” nulli a kΓ—fs/P |
23
- | I nulli cadono a cpp intero indipendentemente da P | **Confermata** β€” P=8, P=16, P=24 collassano tutti a cpp intero |
24
- | L'overlap (stride ridotto) mitiga il nullo | **Rigettata** β€” S=4 collassa quanto S=16 |
25
- | La banda utile del forecaster Γ¨ fs/P | **Confermata** β€” recovery crolla sopra questa soglia |
 
 
 
 
 
 
 
26
 
27
- ## Modelli
28
 
29
- | Cartella | P | S | Overlap | Step | Stato | Ruolo |
30
- |----------|---|---|---------|------|-------|-------|
31
- | `p16-s12-seed42` | 16 | 12 | 25% | 100k | done | Asse stride |
32
- | `p16-s8-seed42` | 16 | 8 | 50% | 100k | done | Asse stride |
33
- | `p16-s4-seed42` | 16 | 4 | 75% | 100k | done | Asse stride |
34
- | `p8-s8-seed42` | 8 | 8 | 0% | 100k | done | Asse patch-size |
35
- | `p24-s24-seed42` | 24 | 24 | 0% | 100k | done | Asse patch-size |
36
- | `p16-s16-seed42` | 16 | 16 | 0% | 10k | **stale** | Sostituito dall'ufficiale |
37
 
38
- > **Nota su p16-s16.** Il retrain p16-s16-seed42 si è fermato a ~33k step e i pesi su HF sono quelli vecchi a 10k. Per il confronto P=16 S=16 usiamo direttamente il modello ufficiale [`amazon/chronos-bolt-tiny`](https://huggingface.co/amazon/chronos-bolt-tiny) (200k step, corpus completo, stessa geometria P=16 S=16). È il miglior anchor possibile per l'asse stride.
39
 
40
- ## Training
 
 
 
 
 
 
 
 
41
 
42
- - **Architettura:** T5-based (Chronos-Bolt Tiny), ~8.65M parametri
43
- - **Dataset:** [autogluon/chronos_datasets](https://huggingface.co/datasets/autogluon/chronos_datasets) β€” TSMixup 10M + KernelSynth 1M, interleaving 9:1
44
- - **Step:** 100,000 (streaming da HF, shuffle buffer 10k)
45
- - **Ottimizzatore:** AdamW fused, LR 1e-3 β†’ 0 (linear decay), no warmup
46
- - **Batch size:** 32, gradient clipping 1.0
47
- - **Precisione:** fp32 + TF32 (RTX 5060 Laptop, sm_120)
48
- - **Context/Prediction:** 2048 / 64
49
 
50
- Ogni modello Γ¨ allenato con lo stesso seed (42), stesso dataset, stessi iperparametri. L'unica variabile Γ¨ la geometria della patch (P, S).
51
-
52
- ## Come usare
53
 
54
  ```python
55
- from chronos import ChronosBoltPipeline
56
- import torch
57
 
58
- # Caricare un singolo modello retrained
59
- pipe = ChronosBoltPipeline.from_pretrained(
60
  "federicosabbadini/chronos-bolt-patch-sweep",
61
- subfolder="p8-s8-seed42",
62
- device_map="cuda",
63
- torch_dtype=torch.float32,
64
- )
65
-
66
- # Caricare il modello ufficiale come reference P=16 S=16
67
- official = ChronosBoltPipeline.from_pretrained(
68
- "amazon/chronos-bolt-tiny",
69
- device_map="cuda",
70
  )
71
-
72
- # Forecast
73
- import numpy as np
74
- context = torch.tensor(np.random.randn(512), dtype=torch.float32)
75
- forecast = pipe.predict(context, prediction_length=64) # [1, 9, 64] β€” 9 quantili
76
- median = forecast[0, 4, :] # quantile 0.5
77
- ```
78
-
79
- ### Caricare tutti i modelli per confronto
80
-
81
- ```python
82
- REPO = "federicosabbadini/chronos-bolt-patch-sweep"
83
- VARIANTS = ["p16-s12-seed42", "p16-s8-seed42", "p16-s4-seed42",
84
- "p8-s8-seed42", "p24-s24-seed42"]
85
-
86
- models = {}
87
- for name in VARIANTS:
88
- models[name] = ChronosBoltPipeline.from_pretrained(
89
- REPO, subfolder=name, device_map="cuda", torch_dtype=torch.float32
90
- )
91
-
92
- # Aggiungere l'ufficiale come p16-s16
93
- models["p16-s16 (official)"] = ChronosBoltPipeline.from_pretrained(
94
- "amazon/chronos-bolt-tiny", device_map="cuda"
95
- )
96
- ```
97
-
98
- ## Struttura del repository
99
-
100
- ```
101
- β”œβ”€β”€ manifest.csv # tabella riassuntiva di tutti i run
102
- β”œβ”€β”€ p16-s12-seed42/
103
- β”‚ β”œβ”€β”€ config.json # config del modello (include P, S)
104
- β”‚ β”œβ”€β”€ model.safetensors # pesi finali (100k step)
105
- β”‚ β”œβ”€β”€ run_config.json # iperparametri del training
106
- β”‚ β”œβ”€β”€ loss_history.npy # curva di loss
107
- β”‚ └── loss_curve.png # plot della loss
108
- β”œβ”€β”€ p16-s8-seed42/ # stessa struttura
109
- β”œβ”€β”€ p16-s4-seed42/
110
- β”œβ”€β”€ p8-s8-seed42/
111
- β”œβ”€β”€ p24-s24-seed42/
112
- └── p16-s16-seed42/ # ⚠️ stale β€” pesi a 10k, non usare
113
  ```
114
 
115
- ## Codice sorgente
116
 
117
- Il codice di training, valutazione e analisi Γ¨ nel repository GitHub: i script di sweep frequenziale, il notebook inspector, e i generatori di segnali sintetici con injection controllata.
 
 
 
 
118
 
119
- ## Licenza
120
 
121
- Apache 2.0 β€” stesso della famiglia Chronos.
 
1
+ # Chronos-Bolt Patch-Stride Sweep
 
 
 
 
 
 
 
 
 
2
 
3
+ Retrained variants of [Chronos-Bolt Tiny](https://huggingface.co/amazon/chronos-bolt-tiny) (~8.7M params) with different **patch size (P)** and **patch stride (S)** configurations, trained from scratch on the full official Chronos pre-training data.
4
 
5
+ ## Purpose
6
 
7
+ These models support the **structural patch-aliasing** study: investigating how the (P, S) patch geometry of Chronos-Bolt creates frequency-dependent artifacts in time-series forecasting. Each model is trained identically except for the patch geometry, so any downstream difference in aliasing probes is attributable to (P, S).
8
 
9
+ ## Models
10
 
11
+ | Subfolder | P | S | Overlap | Status |
12
+ |---|---|---|---|---|
13
+ | `p8-s8-seed42` | 8 | 8 | 0.000 | done |
14
+ | `p16-s4-seed42` | 16 | 4 | 0.750 | done |
15
+ | `p16-s8-seed42` | 16 | 8 | 0.500 | done |
16
+ | `p16-s12-seed42` | 16 | 12 | 0.250 | done |
17
+ | `p16-s16-seed42` | 16 | 16 | 0.000 | done |
18
+ | `p24-s8-seed42` | 24 | 8 | 0.667 | done |
19
+ | `p24-s12-seed42` | 24 | 12 | 0.500 | done |
20
+ | `p24-s16-seed42` | 24 | 16 | 0.333 | done |
21
+ | `p24-s20-seed42` | 24 | 20 | 0.167 | done |
22
+ | `p24-s24-seed42` | 24 | 24 | 0.000 | done |
23
+ | `p32-s32-seed42` | 32 | 32 | 0.000 | done |
24
 
25
+ More models (P=32 stride series + odd-stride S=15 variants) are in training and will be uploaded as they complete.
26
 
27
+ ## Training setup
 
 
 
 
 
 
 
28
 
29
+ All models share the same training regime (only P and S vary):
30
 
31
+ - **Architecture**: Chronos-Bolt Tiny (T5-based, ~8.7M params), random initialization
32
+ - **Data**: Official Chronos pre-training corpus β€” TSMixup (10M series) + KernelSynth (1M series) at 9:1 ratio
33
+ - **Steps**: 100,000 (fixed budget across all runs)
34
+ - **Optimizer**: AdamW, lr=1e-3, linear decay, no warmup
35
+ - **Batch size**: 32
36
+ - **Precision**: fp32 + TF32 matmuls (Ampere+ GPUs)
37
+ - **Context**: 2048 tokens, prediction horizon: 64
38
+ - **Quantiles**: 9 (0.1 to 0.9)
39
+ - **Seed**: 42
40
 
41
+ Full provenance is recorded in each subfolder's `run_config.json`.
 
 
 
 
 
 
42
 
43
+ ## Usage
 
 
44
 
45
  ```python
46
+ from chronos import BaseChronosPipeline
 
47
 
48
+ pipe = BaseChronosPipeline.from_pretrained(
 
49
  "federicosabbadini/chronos-bolt-patch-sweep",
50
+ subfolder="p24-s16-seed42",
51
+ device_map="cpu",
 
 
 
 
 
 
 
52
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  ```
54
 
55
+ ## Files per model
56
 
57
+ - `config.json` β€” model architecture config
58
+ - `model.safetensors` β€” trained weights (~34 MB)
59
+ - `run_config.json` β€” full training provenance + result metrics
60
+ - `loss_history.npy` β€” per-step training loss
61
+ - `loss_curve.png` β€” training loss plot
62
 
63
+ ## License
64
 
65
+ The models are derived from the Chronos architecture (Apache-2.0) and trained on the official Chronos datasets. See the original [Chronos repository](https://github.com/amazon-science/chronos-forecasting) for details.