Add Attention JEPA checkpoints and training logs

f48f919 verified 5 days ago

648 Bytes

license: apache-2.0
tags:
  - jepa
  - attention
  - memory
  - soft-tome

JEPA CMPR Attention checkpoints

Attention-memory JEPA chunk-mask training on Soft-TOME tar_continuation_importance (all shards).

checkpoints/attn-chunkmask-...-allshards/ — early run (step_000000.pt, step_000263.pt)
checkpoints/attn-chunkmask-...-allshards-resume264/ — resume to full schedule (step_000526.pt, step_000789.pt, step_001051.pt final)
logs/ — training logs for both runs

Final checkpoint: checkpoints/attn-chunkmask-Beff256-B64-accum4-K4-lam005-ret0.2-predSA2-t1-ep4-allshards-resume264/step_001051.pt

rookierufus
/

JEPA_CMPR_ATTENTION