turbot2va-ltx2 / code /release /MODEL_CARD.md
luyu1021's picture
Add documented LTX-2 training source, recipes and checkpoint release plan
3da673c verified
|
Raw History Blame Contribute Delete
7.01 kB
---
language:
- en
- zh
license: other
license_name: ltx-2-community
license_link: LICENSE
base_model: Lightricks/LTX-2
tags:
- video-generation
- audio-video
- distillation
- ltx-2
---
# TurboT2VA-LTX2
LTX-2 dev 19B DCM/SCM/RCM/DMD-only research snapshots, curated 2026-11-19. Training source and Chinese annotated recipes are in `code/`. Complete license: `LICENSE`.
## 实际发布范围(2026-11-19)
本 HF 仓库已公开并核验匿名访问;GitHub `liuyuxiang1021/turbot2va-ltx2` 保持私有,HF `code/` 提供相同训练代码与 10 条中文注释配方。
整个筛选包已完成:15 个 DCP 训练 artifact(约 1.322 TB)和 1 个 37.75 GB generator 推理参考,共 232 个权重/元数据文件、1,360,148,787,273 bytes。所有文件已核对远端大小和完整 SHA256;`CHECKSUMS.json` 记录整个集合,`code/release/PUBLISHED_SCOPE.json` 记录权重核验 commit `cda4cffcafa91375d7f43cbd7f26a495227d7af5`。
下表 15 个训练快照均已上传至 `checkpoints/<artifact>/`,其中 4 个包含完整 8-rank optimizer/RNG。已有 generator 参考保留原 stage3/3500 标签,未证明逐 tensor 等同本地 exp5;本次未执行 GPU 推理或重训。下载和推理方法见 `code/docs/INFERENCE.md`,完整训练恢复见 `code/docs/TRAINING.md`。
The curated checkpoints are not LoRA adapters; inference still needs the official dev base assets (AV VAE) and Gemma text encoder. Each published DCP model contains every distributed shard. The four explicitly labeled 8-rank resume artifacts also include optimizer/RNG state. Latest checkpoint does not imply best quality; some snapshots are failure/control trajectories.
## 已发布的 DCP 训练快照清单
下载 DCP 权重时必须完整下载 `sharded/.metadata` 与全部 `.distcp`,同时保留 `metadata.pth`。模型层对联合 RCM/DMD 包含 generator 和 critic;部分完整状态另外含 `optim/` 与 `rng/`。字节数是未压缩源文件大小。
| Artifact | 原分支 / label | completed / next | 上传 GB | 完整续训 | 用途 |
| --- | --- | --- | ---: | --- | --- |
| `dcm_beta0_step2000` | exp1_dcm_8gpu / checkpoint_002000 | 2000 / 2001 | 37.80 | 仅模型层 | DCM beta1=0 warmup parent |
| `dcm_beta0_step2500` | exp1_dcm_8gpu / checkpoint_002500 | 2499 / 2500 | 37.80 | 仅模型层 | DCM beta1=0 end snapshot |
| `dcm_beta09_step2000` | exp4_dcm_beta1_09 / checkpoint_002000 | 2000 / 2001 | 37.80 | 仅模型层 | DCM optimizer A/B parent |
| `dcm_beta09_step2500` | exp4_dcm_beta1_09 / checkpoint_002500 | 2499 / 2500 | 37.80 | 仅模型层 | exp6 warmup parent |
| `scm_prefix_step500` | exp3_dcm2000_scm500 / checkpoint_000500 | 499 / 500 | 113.31 | 8卡,含 optimizer/RNG | Common SCM parent for exp5 / DMD-only / SCM continuation |
| `rcm_scm500_step2000` | exp5_dcm2000_scm500_rcm / checkpoint_002000 | 2000 / 2001 | 75.60 | 仅模型层 | Earlier evaluated RCM comparison point |
| `rcm_scm500_step3500` | exp5_dcm2000_scm500_rcm / checkpoint_003500 | 3500 / 3501 | 226.62 | 8卡,含 optimizer/RNG | Latest exp5 training checkpoint |
| `rcm_dcm2500_step3000` | exp6_dcm2500_beta09_rcm / checkpoint_003000 | 3000 / 3001 | 75.60 | 仅模型层 | Earlier evaluated snapshot before later score drops |
| `rcm_dcm2500_step5000` | exp6_dcm2500_beta09_rcm / checkpoint_005000 | 5000 / 5001 | 226.62 | 8卡,含 optimizer/RNG | Latest exp6 training checkpoint, not asserted best |
| `dmd_only_step1525` | dcm2000_scm500_dmdonly_20261117 / checkpoint_001525 | 1524 / 1525 | 226.62 | 8卡,含 optimizer/RNG | Safely stopped DMD-only baseline with evaluated snapshot |
| `rcm_direct_step3504` | direct_rcm_retrain_20261116 / checkpoint_003504 | 3503 / 3504 | 75.60 | 仅模型层 | Latest safely stopped direct RCM baseline |
| `scm_direct_beta0_step1000` | exp2_scm_8gpu / checkpoint_001000 | 1000 / 1001 | 37.80 | 仅模型层 | SCM from base beta1=0 failure/control trajectory |
| `scm_direct_beta09_step1619` | direct_scm_beta09_20261116 / checkpoint_001619 | 1618 / 1619 | 37.80 | 仅模型层 | Latest direct SCM beta1=0.9 control |
| `scm_continue_step1000` | b_scm_only_cont_20261116 / checkpoint_001000 | 1000 / 1001 | 37.80 | 仅模型层 | Evaluated common-prefix SCM-only baseline |
| `scm_continue_step2119` | b_scm_only_cont_20261116 / checkpoint_002119 | 2118 / 2119 | 37.80 | 仅模型层 | Latest common-prefix SCM-only continuation |
关键 DCP 总计约 1.32 TB;原训练目录 56 个、91 个 checkpoint 共约 11.635 TB,中间产物留在本地。
`rcm_scm500_step3500` 与 `rcm_dcm2500_step5000` 是主线最新恢复版本;较早评测版本同时保留。`dmd_only_step1525` 有同编号评测;`rcm_direct_step3504` 是安全停点,其最近成套评测为 3500;`scm_direct_beta09_step1619`、`scm_continue_step2119` 是停止版本,不能把早期/其他编号评测算到该步。
另保留已有发布推理参考 `inference/published_turbot2av_main/model.pth`,约 37.754 GB、3510 个 BF16 tensor,只有 generator。其随附 `checkpoint_info.json` 记录 stage3/3500 与 SHA256 `ef31ea8334cc02c8f65da8c5c66b634eb594bca4a827b5cbbd72d390ffe27c61`;此记录未证明它等同本地 exp5 checkpoint,不合并其来源。
源 checkpoint 完整性已检查 metadata、非空模型 shard 数、optimizer/RNG 数。模型文件的内容 SHA256 由上传前逐文件读取生成,上传成功后用远端哈希核验;此前的结构检查不冒充全文件 hash 验证。
完整 resume 要求相同 world size、参数布局、依赖环境、可用数据/基座;模型层初始化不继承 optimizer。原始元数据保持完整,恢复范围以每份 RELEASE_METADATA.json 的 full_resume_included 为准。
## Inference
Use `code/LTX-2` pinned PyTorch 2.7.0+cu128 environment. See `code/README.md`, `code/docs/TRAINING.md`, and `code/docs/EVALUATION.md`. Use `inference/published_turbot2av_main/model.pth` for the existing generator reference, or download all shards under a chosen `checkpoints/<artifact>/` for DCP inference. The inference runner accepts either the generator file or a complete DCP directory. Follow `code/docs/INFERENCE.md`. The generator reference cannot recover optimizer training.
## Evaluation and limitations
Training probes (8 prompts x 1 seed) are distinct from formal 200 prompts x 8 seeds diversity. Raw and enhanced prompt protocols are recorded separately. Local raw200 prompts have 16 exact textual overlaps with training mapping. Javis/AVH are AV similarity measures, not full subjective quality. exp6 later snapshots show lower AV scores; earlier 3000 and latest 5000 are both published. No new large scale or human evaluation was run for this release.
## Upload completion
The full selected plan is uploaded and verified. `CHECKSUMS.json` covers all 232 published checkpoint and release metadata files; `code/release/PUBLISHED_SCOPE.json` records the verified scope. Source code can be updated in later commits while retaining the weight verification revision.