| license: apache-2.0 | |
| tags: | |
| - jepa | |
| - attention | |
| - memory | |
| - soft-tome | |
| # JEPA CMPR Attention checkpoints | |
| Attention-memory JEPA chunk-mask training on Soft-TOME `tar_continuation_importance` (all shards). | |
| ## Contents | |
| - `checkpoints/attn-chunkmask-...-allshards/` — early run (`step_000000.pt`, `step_000263.pt`) | |
| - `checkpoints/attn-chunkmask-...-allshards-resume264/` — resume to full schedule (`step_000526.pt`, `step_000789.pt`, `step_001051.pt` final) | |
| - `logs/` — training logs for both runs | |
| Final checkpoint: `checkpoints/attn-chunkmask-Beff256-B64-accum4-K4-lam005-ret0.2-predSA2-t1-ep4-allshards-resume264/step_001051.pt` | |