Commit History
21 days ago
-
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signatureb8fa788 -
Training finished: 29 of 30 checkpoints; Gemma protect run 1 cancelled (not trained)
Verified signatureAt Josh's request (Oct 16, 3:11 PM PT) no further protect runs were trained; Gemma protect run 1 was still waiting for a GPU machine and is listed as not trained. Corrects the earlier allocator note: only facts run 1 and protect run 3 were relaunched with expandable_segments.acb2a1b -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature09b9a46 -
Record the allocator setting (expandable_segments) of three Gemma RunPod runs; current launch script
Verified signatureGemma facts run 1 and protect runs 1 and 3 OOMed on RunPod from allocator fragmentation and were relaunched with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (already set by the Slurm job template). Allocator setting only; no change to the computation.df9bf6f -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature1a48ed6 -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature57a16f0 -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature54006fd -
Refresh runs.csv, checkpoint table and release log (sync_release.py)
Verified signature95b871d -
Docs: weights now live in each checkpoint's model repo; folders here hold configs and WEIGHTS.json
Verified signature4b2d2e2 -
Restore release docs, data and per-run configs without the duplicated weights
Verified signatureThe release repo previously held byte-identical copies of every checkpoint's weights (1.02 TB), which pushed the account over its public storage limit on 2026-10-15. The weights stay in the per-run model repos; this repo now links to them. All 292 non-weight files are restored unchanged.9928897