Pre-train and Mid-train
This repository contains the full pre-train checkpoint and the PICO mid-train checkpoint. The mid-train model was initialized directly from the pre-train model. Each checkpoint has its resolved training config.yaml and dataset_stats.json.
| Stage | Checkpoint |
|---|---|
| Full pre-train | pretrain/step_037617.pt |
| PICO mid-train | midtrain/step_020560.pt |
Download:
hf download WB-WAM/WB-WAM-Pretrain-Midtrain --local-dir ./checkpoints/wbwam
Training code and instructions are in the WB-WAM repository.