turbot2va-ltx2 / code /docs /CHECKPOINTS.md
luyu1021's picture
Add documented LTX-2 training source, recipes and checkpoint release plan
3da673c verified
|
Raw History Blame Contribute Delete
4.4 kB

关键 checkpoint 与用途

2026-11-19 实际发布范围:HF 已公开,37.75 GB generator 推理参考及下表 15 个 DCP 训练快照全部上传并通过大小/SHA256 核验,共 232 个权重/元数据文件。其中 4 个快照包含完整 8-rank optimizer/RNG。实际记录见 PUBLISHED_SCOPE.json,当前权重使用见 INFERENCE.md。

所有 DCP 权重都必须完整下载 sharded/.metadata 与全部 .distcp,同时保留 metadata.pth。模型层对联合 RCM/DMD 包含 generator 和 critic;部分完整状态另外含 optim/ 与 rng/。字节数是未压缩源文件大小。

Artifact 原分支 / label completed / next 上传 GB 完整续训 用途
dcm_beta0_step2000 exp1_dcm_8gpu / checkpoint_002000 2000 / 2001 37.80 仅模型层 DCM beta1=0 warmup parent
dcm_beta0_step2500 exp1_dcm_8gpu / checkpoint_002500 2499 / 2500 37.80 仅模型层 DCM beta1=0 end snapshot
dcm_beta09_step2000 exp4_dcm_beta1_09 / checkpoint_002000 2000 / 2001 37.80 仅模型层 DCM optimizer A/B parent
dcm_beta09_step2500 exp4_dcm_beta1_09 / checkpoint_002500 2499 / 2500 37.80 仅模型层 exp6 warmup parent
scm_prefix_step500 exp3_dcm2000_scm500 / checkpoint_000500 499 / 500 113.31 8卡,含 optimizer/RNG Common SCM parent for exp5 / DMD-only / SCM continuation
rcm_scm500_step2000 exp5_dcm2000_scm500_rcm / checkpoint_002000 2000 / 2001 75.60 仅模型层 Earlier evaluated RCM comparison point
rcm_scm500_step3500 exp5_dcm2000_scm500_rcm / checkpoint_003500 3500 / 3501 226.62 8卡,含 optimizer/RNG Latest exp5 training checkpoint
rcm_dcm2500_step3000 exp6_dcm2500_beta09_rcm / checkpoint_003000 3000 / 3001 75.60 仅模型层 Earlier evaluated snapshot before later score drops
rcm_dcm2500_step5000 exp6_dcm2500_beta09_rcm / checkpoint_005000 5000 / 5001 226.62 8卡,含 optimizer/RNG Latest exp6 training checkpoint, not asserted best
dmd_only_step1525 dcm2000_scm500_dmdonly_20261117 / checkpoint_001525 1524 / 1525 226.62 8卡,含 optimizer/RNG Safely stopped DMD-only baseline with evaluated snapshot
rcm_direct_step3504 direct_rcm_retrain_20261116 / checkpoint_003504 3503 / 3504 75.60 仅模型层 Latest safely stopped direct RCM baseline
scm_direct_beta0_step1000 exp2_scm_8gpu / checkpoint_001000 1000 / 1001 37.80 仅模型层 SCM from base beta1=0 failure/control trajectory
scm_direct_beta09_step1619 direct_scm_beta09_20261116 / checkpoint_001619 1618 / 1619 37.80 仅模型层 Latest direct SCM beta1=0.9 control
scm_continue_step1000 b_scm_only_cont_20261116 / checkpoint_001000 1000 / 1001 37.80 仅模型层 Evaluated common-prefix SCM-only baseline
scm_continue_step2119 b_scm_only_cont_20261116 / checkpoint_002119 2118 / 2119 37.80 仅模型层 Latest common-prefix SCM-only continuation

关键 DCP 总计约 1.32 TB;原训练目录 56 个、91 个 checkpoint 共约 11.635 TB,中间产物留在本地。

rcm_scm500_step3500 与 rcm_dcm2500_step5000 是主线最新恢复版本;较早评测版本同时保留。dmd_only_step1525 有同编号评测;rcm_direct_step3504 是安全停点,其最近成套评测为 3500;scm_direct_beta09_step1619、scm_continue_step2119 是停止版本,不能把早期/其他编号评测算到该步。

另保留已有发布推理参考 inference/published_turbot2av_main/model.pth,约 37.754 GB、3510 个 BF16 tensor,只有 generator。其随附 checkpoint_info.json 记录 stage3/3500 与 SHA256 ef31ea8334cc02c8f65da8c5c66b634eb594bca4a827b5cbbd72d390ffe27c61;此记录未证明它等同本地 exp5 checkpoint,不合并其来源。

源 checkpoint 完整性已检查 metadata、非空模型 shard 数、optimizer/RNG 数。模型文件的内容 SHA256 由上传前逐文件读取生成,上传成功后用远端哈希核验;此前的结构检查不冒充全文件 hash 验证。

完整 resume 要求相同 world size、参数布局、依赖环境、可用数据/基座;模型层初始化不继承 optimizer。原始元数据保持完整,恢复范围以每份 RELEASE_METADATA.json 的 full_resume_included 为准。