stamp-test / DATASET.md
selen-kim's picture
Upload DATASET.md with huggingface_hub
aa802e1 verified
|
Raw
History Blame Contribute Delete
1.27 kB
# Dataset layout
All data files are stored physically under `dataset/`. There are no symlinks.
| Path | Meaning | Consumed by |
|---|---|---|
| `dataset/source_csv/` | Original clustered CSV traces for 20 devices | pair/outpaint precompute and raw evaluation |
| `dataset/windows_v5/` | Time-ordered 256-token windows before V2 tokenization | V2 conversion and condition reconstruction |
| `dataset/tokenized_v2/` | VAE-ready DELTA-256 + RETURN-32 representation | VAE training and decoding |
| `dataset/precomputed/singlegen/` | VAE latents plus 7-D conditions for individual windows | single-window DiT |
| `dataset/precomputed/outgen/` | Prefix/suffix latent pairs plus 14-D conditions | outpaint DiT |
The flow is:
```text
source_csv ────────────────┐
β”œβ”€> precomputed/outgen ─> checkpoints/outgen
windows_v5 ─> tokenized_v2─
└─> precomputed/singlegen ─> checkpoints/singlegen
└────────> checkpoints/vae
```
`precomputed` is still data, not a model checkpoint. It contains frozen VAE
latent tensors used to avoid encoding all windows again during DiT training.
The actual learned weights live only under `checkpoints/`.