Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- time-series
|
| 5 |
+
- forecasting
|
| 6 |
+
- chronos
|
| 7 |
+
- patch-aliasing
|
| 8 |
+
language:
|
| 9 |
+
- en
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Chronos-Bolt Patch Sweep
|
| 13 |
+
|
| 14 |
+
A collection of **Chronos-Bolt Tiny** models retrained **from scratch** (random weights) on the official [Chronos pre-training corpus](https://huggingface.co/datasets/autogluon/chronos_datasets) (TSMixup 10M + KernelSynth 1M), varying only the **patch geometry** (input_patch_size `P` and input_patch_stride `S`).
|
| 15 |
+
|
| 16 |
+
The goal is to study **structural aliasing** in patched time-series transformers: how the choice of patch size and stride affects the model's ability to represent different frequency components.
|
| 17 |
+
|
| 18 |
+
## Models
|
| 19 |
+
|
| 20 |
+
| Subfolder | P | S | Overlap Ratio | Approx Patches | Params | Final Loss | Speed (it/s) |
|
| 21 |
+
|-----------|---|---|---------------|-----------------|--------|------------|--------------|
|
| 22 |
+
| `p16-s16-seed42` | 16 | 16 | 0.00 (contiguous) | 128 | 8.65M | 36.09 | 4.94 |
|
| 23 |
+
| `p16-s12-seed42` | 16 | 12 | 0.25 | 170 | 8.65M | 35.34 | 6.78 |
|
| 24 |
+
| `p16-s8-seed42` | 16 | 8 | 0.50 | 255 | 8.65M | 36.97 | 3.80 |
|
| 25 |
+
| `p16-s4-seed42` | 16 | 4 | 0.75 | 509 | 8.65M | 36.70 | 3.58 |
|
| 26 |
+
| `p8-s8-seed42` | 8 | 8 | 0.00 (contiguous) | 256 | 8.63M | 37.62 | 7.03 |
|
| 27 |
+
| `p24-s24-seed42` | 24 | 24 | 0.00 (contiguous) | 86 | 8.67M | 34.90 | 4.50 |
|
| 28 |
+
|
| 29 |
+
### Experimental design
|
| 30 |
+
|
| 31 |
+
- **Overlap axis** (fixed P=16, varying S): `S=16` (no overlap) β `S=12` (25%) β `S=8` (50%) β `S=4` (75%). Tests whether overlapping patches reduce aliasing artifacts.
|
| 32 |
+
- **Patch-size axis** (contiguous, S=P): `P=8` β `P=16` β `P=24`. Tests how the patch size itself (the effective sampling window) affects frequency representation.
|
| 33 |
+
|
| 34 |
+
All other hyperparameters are fixed at the official Chronos-Bolt Tiny values so any downstream difference is attributable to P/S alone.
|
| 35 |
+
|
| 36 |
+
## Training details
|
| 37 |
+
|
| 38 |
+
- **Base architecture**: [amazon/chronos-bolt-tiny](https://huggingface.co/amazon/chronos-bolt-tiny) (T5-based, ~8.65M params)
|
| 39 |
+
- **Weights**: trained from scratch (random init), NOT fine-tuned from the official checkpoint
|
| 40 |
+
- **Dataset**: `autogluon/chronos_datasets` β TSMixup 10M + KernelSynth 1M (official Chronos corpus)
|
| 41 |
+
- **Steps**: 10,000 (vs official 200k β reduced compute budget; all models compared only against each other)
|
| 42 |
+
- **Batch size**: 32
|
| 43 |
+
- **Optimizer**: AdamW (fused), LR=1e-3, linear decay, no warmup
|
| 44 |
+
- **Precision**: fp32 + TF32 matmuls on NVIDIA RTX 5060 Laptop GPU
|
| 45 |
+
- **Context length**: 2048 tokens
|
| 46 |
+
- **Prediction length**: 64 steps
|
| 47 |
+
- **Quantiles**: 9 (0.1 to 0.9)
|
| 48 |
+
|
| 49 |
+
## Usage
|
| 50 |
+
|
| 51 |
+
```python
|
| 52 |
+
from chronos import ChronosBoltPipeline
|
| 53 |
+
import torch
|
| 54 |
+
|
| 55 |
+
# Load any single model
|
| 56 |
+
pipeline = ChronosBoltPipeline.from_pretrained(
|
| 57 |
+
"federicosabbadini/chronos-bolt-patch-sweep",
|
| 58 |
+
subfolder="p16-s16-seed42",
|
| 59 |
+
device_map="cuda",
|
| 60 |
+
torch_dtype=torch.float32,
|
| 61 |
+
)
|
| 62 |
+
|
| 63 |
+
# Forecast
|
| 64 |
+
context = torch.tensor([...]) # your time series as a 1D tensor
|
| 65 |
+
forecast = pipeline.predict(context, prediction_length=64)
|
| 66 |
+
# forecast shape: (1, num_samples, 64)
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
### Load all models for comparison
|
| 70 |
+
|
| 71 |
+
```python
|
| 72 |
+
from chronos import ChronosBoltPipeline
|
| 73 |
+
import torch
|
| 74 |
+
|
| 75 |
+
REPO = "federicosabbadini/chronos-bolt-patch-sweep"
|
| 76 |
+
MODEL_NAMES = [
|
| 77 |
+
"p16-s16-seed42", "p16-s12-seed42", "p16-s8-seed42",
|
| 78 |
+
"p16-s4-seed42", "p8-s8-seed42", "p24-s24-seed42",
|
| 79 |
+
]
|
| 80 |
+
|
| 81 |
+
models = {}
|
| 82 |
+
for name in MODEL_NAMES:
|
| 83 |
+
models[name] = ChronosBoltPipeline.from_pretrained(
|
| 84 |
+
REPO, subfolder=name,
|
| 85 |
+
device_map="cuda", torch_dtype=torch.float32,
|
| 86 |
+
)
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
## Files per model
|
| 90 |
+
|
| 91 |
+
Each subfolder contains:
|
| 92 |
+
- `config.json` β HuggingFace model config (includes `chronos_config` with P, S values)
|
| 93 |
+
- `model.safetensors` β trained weights (~33 MB)
|
| 94 |
+
- `run_config.json` β full provenance + training result metadata
|
| 95 |
+
- `loss_history.npy` β per-step training loss array (10,000 values)
|
| 96 |
+
- `loss_curve.png` β training loss plot
|
| 97 |
+
|
| 98 |
+
The root `manifest.csv` summarizes all finished runs.
|
| 99 |
+
|
| 100 |
+
## Citation
|
| 101 |
+
|
| 102 |
+
This work is part of a research project on patch aliasing in time-series transformers. The base model architecture and training pipeline are adapted from [Chronos](https://github.com/amazon-science/chronos-forecasting) (Apache-2.0).
|