Commit History
23 days ago
-
Training finished: 29 of 30 checkpoints; Gemma protect run 1 cancelled (not trained)
Verified signatureAt Josh's request (Oct 16, 3:11 PM PT) no further protect runs were trained; Gemma protect run 1 was still waiting for a GPU machine and is listed as not trained. Corrects the earlier allocator note: only facts run 1 and protect run 3 were relaunched with expandable_segments.acb2a1b -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature09b9a46 -
Record the allocator setting (expandable_segments) of three Gemma RunPod runs; current launch script
Verified signatureGemma facts run 1 and protect runs 1 and 3 OOMed on RunPod from allocator fragmentation and were relaunched with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (already set by the Slurm job template). Allocator setting only; no change to the computation.df9bf6f -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature1a48ed6 -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature57a16f0 -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature54006fd -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature95b871d -
Restore release docs, data and per-run configs without the duplicated weights (part 2)
Verified signatureThe release repo previously held byte-identical copies of every checkpoint's weights (1.02 TB), which pushed the account over its public storage limit on 2026-10-15. The weights stay in the per-run model repos; this repo now links to them. All 292 non-weight files are restored unchanged.53b43be