federicosabbadini commited on
Commit
3d51337
Β·
verified Β·
1 Parent(s): 67043d1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - time-series
5
+ - forecasting
6
+ - chronos
7
+ - patch-aliasing
8
+ language:
9
+ - en
10
+ ---
11
+
12
+ # Chronos-Bolt Patch Sweep
13
+
14
+ A collection of **Chronos-Bolt Tiny** models retrained **from scratch** (random weights) on the official [Chronos pre-training corpus](https://huggingface.co/datasets/autogluon/chronos_datasets) (TSMixup 10M + KernelSynth 1M), varying only the **patch geometry** (input_patch_size `P` and input_patch_stride `S`).
15
+
16
+ The goal is to study **structural aliasing** in patched time-series transformers: how the choice of patch size and stride affects the model's ability to represent different frequency components.
17
+
18
+ ## Models
19
+
20
+ | Subfolder | P | S | Overlap Ratio | Approx Patches | Params | Final Loss | Speed (it/s) |
21
+ |-----------|---|---|---------------|-----------------|--------|------------|--------------|
22
+ | `p16-s16-seed42` | 16 | 16 | 0.00 (contiguous) | 128 | 8.65M | 36.09 | 4.94 |
23
+ | `p16-s12-seed42` | 16 | 12 | 0.25 | 170 | 8.65M | 35.34 | 6.78 |
24
+ | `p16-s8-seed42` | 16 | 8 | 0.50 | 255 | 8.65M | 36.97 | 3.80 |
25
+ | `p16-s4-seed42` | 16 | 4 | 0.75 | 509 | 8.65M | 36.70 | 3.58 |
26
+ | `p8-s8-seed42` | 8 | 8 | 0.00 (contiguous) | 256 | 8.63M | 37.62 | 7.03 |
27
+ | `p24-s24-seed42` | 24 | 24 | 0.00 (contiguous) | 86 | 8.67M | 34.90 | 4.50 |
28
+
29
+ ### Experimental design
30
+
31
+ - **Overlap axis** (fixed P=16, varying S): `S=16` (no overlap) β†’ `S=12` (25%) β†’ `S=8` (50%) β†’ `S=4` (75%). Tests whether overlapping patches reduce aliasing artifacts.
32
+ - **Patch-size axis** (contiguous, S=P): `P=8` β†’ `P=16` β†’ `P=24`. Tests how the patch size itself (the effective sampling window) affects frequency representation.
33
+
34
+ All other hyperparameters are fixed at the official Chronos-Bolt Tiny values so any downstream difference is attributable to P/S alone.
35
+
36
+ ## Training details
37
+
38
+ - **Base architecture**: [amazon/chronos-bolt-tiny](https://huggingface.co/amazon/chronos-bolt-tiny) (T5-based, ~8.65M params)
39
+ - **Weights**: trained from scratch (random init), NOT fine-tuned from the official checkpoint
40
+ - **Dataset**: `autogluon/chronos_datasets` β€” TSMixup 10M + KernelSynth 1M (official Chronos corpus)
41
+ - **Steps**: 10,000 (vs official 200k β€” reduced compute budget; all models compared only against each other)
42
+ - **Batch size**: 32
43
+ - **Optimizer**: AdamW (fused), LR=1e-3, linear decay, no warmup
44
+ - **Precision**: fp32 + TF32 matmuls on NVIDIA RTX 5060 Laptop GPU
45
+ - **Context length**: 2048 tokens
46
+ - **Prediction length**: 64 steps
47
+ - **Quantiles**: 9 (0.1 to 0.9)
48
+
49
+ ## Usage
50
+
51
+ ```python
52
+ from chronos import ChronosBoltPipeline
53
+ import torch
54
+
55
+ # Load any single model
56
+ pipeline = ChronosBoltPipeline.from_pretrained(
57
+ "federicosabbadini/chronos-bolt-patch-sweep",
58
+ subfolder="p16-s16-seed42",
59
+ device_map="cuda",
60
+ torch_dtype=torch.float32,
61
+ )
62
+
63
+ # Forecast
64
+ context = torch.tensor([...]) # your time series as a 1D tensor
65
+ forecast = pipeline.predict(context, prediction_length=64)
66
+ # forecast shape: (1, num_samples, 64)
67
+ ```
68
+
69
+ ### Load all models for comparison
70
+
71
+ ```python
72
+ from chronos import ChronosBoltPipeline
73
+ import torch
74
+
75
+ REPO = "federicosabbadini/chronos-bolt-patch-sweep"
76
+ MODEL_NAMES = [
77
+ "p16-s16-seed42", "p16-s12-seed42", "p16-s8-seed42",
78
+ "p16-s4-seed42", "p8-s8-seed42", "p24-s24-seed42",
79
+ ]
80
+
81
+ models = {}
82
+ for name in MODEL_NAMES:
83
+ models[name] = ChronosBoltPipeline.from_pretrained(
84
+ REPO, subfolder=name,
85
+ device_map="cuda", torch_dtype=torch.float32,
86
+ )
87
+ ```
88
+
89
+ ## Files per model
90
+
91
+ Each subfolder contains:
92
+ - `config.json` β€” HuggingFace model config (includes `chronos_config` with P, S values)
93
+ - `model.safetensors` β€” trained weights (~33 MB)
94
+ - `run_config.json` β€” full provenance + training result metadata
95
+ - `loss_history.npy` β€” per-step training loss array (10,000 values)
96
+ - `loss_curve.png` β€” training loss plot
97
+
98
+ The root `manifest.csv` summarizes all finished runs.
99
+
100
+ ## Citation
101
+
102
+ This work is part of a research project on patch aliasing in time-series transformers. The base model architecture and training pipeline are adapted from [Chronos](https://github.com/amazon-science/chronos-forecasting) (Apache-2.0).