Add CMD model card and checkpoints

README.md

@@ -0,0 +1,176 @@
+---
+license: other
+license_name: nvidia-oneway-noncommercial
+license_link: https://github.com/nv-tlabs/cmd/blob/main/LICENSE
+pipeline_tag: image-to-video
+tags:
+  - video-generation
+  - image-to-video
+  - world-model
+  - diffusion
+  - autoregressive
+  - camera-control
+  - safetensors
+---
+
+# Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation
+
+**Hmrishav Bandyopadhyay<sup>1,2</sup>, Xuanchi Ren<sup>1</sup>, Zijian Huang<sup>1</sup>, Jay Zhangjie Wu<sup>1</sup>, Tianshi Cao<sup>1</sup>, Ruilong Li<sup>1</sup>, Bryan Chu<sup>1</sup>, Sanja Fidler<sup>1</sup>, Yi-Zhe Song<sup>2</sup>, Zian Wang<sup>1</sup>**
+
+<sup>1</sup>NVIDIA  
+<sup>2</sup>SketchX, CVSSP, University of Surrey
+
+[Project Page](https://hmrishavbandy.github.io/cmd-site/) | [Code](https://github.com/nv-tlabs/cmd)
+
+## Model Overview
+
+### Description
+
+Context-Matched Distillation (CMD) is a method for autoregressive video distillation using causal teachers. This repository provides CMD checkpoints based on Cosmos-Predict2.5 2B for causal image-to-video generation.
+
+The release includes chunk-1 and chunk-4 models for short- and long-duration generation, together with camera-control variants. The models use local temporal attention and generate video autoregressively in causal chunks.
+
+This model is for research and development only.
+
+### License and Terms of Use
+
+This model is released under the [NVIDIA OneWay Noncommercial License](https://github.com/NVIDIA/generative-model-design-license/blob/main/NVIDIA%20OneWay%20Noncommercial%20License.md). It may only be used for non-commercial research or educational purposes.
+
+### Deployment Geography
+
+Global
+
+### Intended Use
+
+Primary users include researchers and developers studying:
+
+- Autoregressive video generation.
+- Causal video distillation.
+- Long-duration video generation.
+- Interactive world models.
+- Camera-controlled video generation.
+
+The checkpoints are not intended for safety-critical applications or uses prohibited by the model license.
+
+## Available Checkpoints
+
+| Checkpoint | Configuration | Intended use |
+|---|---|---|
+| `chunk1_short_t24_l21.safetensors` | `t24_l21` | Chunk-1 short-duration generation |
+| `chunk4_short_t21_l16.safetensors` | `t21_l16` | Chunk-4 short-duration generation |
+| `chunk1_long_t126_l21.safetensors` | `t126_l21` | Chunk-1 long-duration generation |
+| `chunk4_long_t121_l16.safetensors` | `t121_l16` | Chunk-4 long-duration generation |
+| `chunk1_camera_control_t32_l21.safetensors` | `t32_l21` | Chunk-1 generation with camera control |
+| `chunk4_camera_control_t29_l24.safetensors` | `t29_l24` | Chunk-4 generation with camera control |
+
+In the configuration names, `t` denotes frame duration and `l` denotes local attention. Checkpoint names do not include dataset names or training iteration information.
+
+## Quick Start
+
+Install the reference implementation:
+
+```bash
+git clone https://github.com/nv-tlabs/cmd.git
+cd cmd
+
+conda create -n causal-cosmos python=3.10 -y
+conda activate causal-cosmos
+python -m pip install -r requirements.txt
+python -m pip install flash-attn --no-build-isolation
+python setup.py develop
+```
+
+Download the checkpoints:
+
+```bash
+hf download nvidia/cmd --local-dir checkpoints
+```
+
+To download one model only:
+
+```bash
+hf download nvidia/cmd \
+  chunk1_short_t24_l21.safetensors \
+  --local-dir checkpoints
+```
+
+Follow the inference instructions in the [CMD repository](https://github.com/nv-tlabs/cmd) to select the matching model configuration and run generation.
+
+## Model Architecture
+
+Architecture type: Causal video diffusion transformer
+
+Base model: [Cosmos-Predict2.5 2B](https://huggingface.co/nvidia/Cosmos-Predict2.5-2B)
+
+Generation strategy: Block-autoregressive video generation
+
+CMD uses causal temporal attention to generate video sequentially. Chunk-1 and chunk-4 variants provide different autoregressive generation granularities, while the camera-control variants add camera-motion conditioning.
+
+## Inputs
+
+Standard checkpoints accept:
+
+- An initial image.
+- A text prompt.
+
+Camera-control checkpoints additionally accept a camera trajectory in the format expected by the CMD inference pipeline.
+
+## Outputs
+
+The model produces an RGB video conditioned on the initial image, text prompt, and, when applicable, camera trajectory. Output duration is determined by the selected `t` configuration.
+
+## Software Integration
+
+- Framework: PyTorch
+- Checkpoint format: Safetensors
+- Reference implementation: [nv-tlabs/cmd](https://github.com/nv-tlabs/cmd)
+- Preferred operating system: Linux
+- Recommended hardware: NVIDIA CUDA-capable GPU
+
+The checkpoints must be used with the corresponding model architecture and configuration supplied by the reference implementation.
+
+## Training Method
+
+CMD uses a two-stage procedure:
+
+1. Causal teacher pretraining.
+2. Context-matched distillation.
+
+See the [CMD repository](https://github.com/nv-tlabs/cmd) and [project page](https://hmrishavbandy.github.io/cmd-site/) for the method and evaluation details.
+
+## Limitations
+
+- Autoregressive errors may accumulate during long rollouts.
+- Visual quality and temporal consistency may degrade over time.
+- Generated motion may be physically implausible.
+- Fine details, text, faces, hands, and rapid motion may contain artifacts.
+- Prompt adherence may vary with scene complexity.
+- Camera-control outputs may deviate from the requested trajectory.
+- Generated content may reflect biases present in the training data.
+
+Users should evaluate the model on data representative of their intended application before deployment.
+
+## Ethical Considerations
+
+Generated video may be mistaken for authentic footage. Users should clearly identify synthetic content where appropriate and must not use the model for deception, impersonation, harassment, misinformation, or violations of privacy, intellectual-property, or other applicable rights.
+
+NVIDIA believes Trustworthy AI is a shared responsibility. Developers should ensure that their use of this model complies with applicable laws, policies, licenses, and safety requirements.
+
+Please report model quality, risk, security vulnerabilities, or NVIDIA AI concerns through the [NVIDIA support portal](https://www.nvidia.com/en-us/support/submit-a-case/).
+
+## Citation
+
+If you find this work useful, please cite:
+
+```bibtex
+@misc{bandyopadhyay2026contextmatched,
+  title  = {Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation},
+  author = {Bandyopadhyay, Hmrishav and Ren, Xuanchi and Huang, Zijian and Wu, Jay Zhangjie and Cao, Tianshi and Li, Ruilong and Chu, Bryan and Fidler, Sanja and Song, Yi-Zhe and Wang, Zian},
+  year   = {2026},
+  url    = {https://hmrishavbandy.github.io/cmd-site/}
+}
+```
+
+## Acknowledgements
+
+This project builds on [Self-Forcing](https://github.com/guandeh17/Self-Forcing) and [NVIDIA Cosmos-Predict2.5](https://github.com/nvidia-cosmos/cosmos-predict2.5).

chunk1_camera_control_t32_l21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4aaefa51da60af7b145f3f879f4261f1c2f15bbaf215d5de90776d6c146b907
+size 4294578646

chunk1_camera_control_t32_l21.safetensors.sha256

@@ -0,0 +1 @@
+f4aaefa51da60af7b145f3f879f4261f1c2f15bbaf215d5de90776d6c146b907  chunk1_camera_control_t32_l21.safetensors

chunk1_long_t126_l21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc719a26615662bb3cefd99317a90184c6e9e31595a58181e1ef37cc9e144d13
+size 8236764356

chunk1_long_t126_l21.safetensors.sha256

@@ -0,0 +1 @@
+bc719a26615662bb3cefd99317a90184c6e9e31595a58181e1ef37cc9e144d13  chunk1_long_t126_l21.safetensors

chunk1_short_t24_l21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18222c475e62bec82aa61c7e963a1d2dc8e9cc97dc5a7758cc5f1bdd439ada65
+size 8236764356

chunk1_short_t24_l21.safetensors.sha256

@@ -0,0 +1 @@
+18222c475e62bec82aa61c7e963a1d2dc8e9cc97dc5a7758cc5f1bdd439ada65  chunk1_short_t24_l21.safetensors

chunk1_teacher_t24_l21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d26644d8e649934317d996331d1a4b5ebfe046e9a1f4c2cfcd413014a3d37df7
+size 8236764356

chunk1_teacher_t24_l21.safetensors.sha256

@@ -0,0 +1 @@
+d26644d8e649934317d996331d1a4b5ebfe046e9a1f4c2cfcd413014a3d37df7  chunk1_teacher_t24_l21.safetensors

chunk1_teacher_t32_l21_camera.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3111f51b326eb518148264d381b18dc880ad42d4246a2cf437961227b50957e8
+size 8589089124

chunk1_teacher_t32_l21_camera.safetensors.sha256

@@ -0,0 +1 @@
+3111f51b326eb518148264d381b18dc880ad42d4246a2cf437961227b50957e8  chunk1_teacher_t32_l21_camera.safetensors

chunk4_camera_control_t29_l24.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e512bd564018098244aa0c867b2f49862016a651b770e1e04e469cfaf43309ff
+size 4294578646

chunk4_camera_control_t29_l24.safetensors.sha256

@@ -0,0 +1 @@
+e512bd564018098244aa0c867b2f49862016a651b770e1e04e469cfaf43309ff  chunk4_camera_control_t29_l24.safetensors

chunk4_long_t121_l16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d00a8f536bb69c5f9d6d2b9571e437c7ee5427299755ba1cecab56daeda6d8a
+size 8236764356

chunk4_long_t121_l16.safetensors.sha256

@@ -0,0 +1 @@
+4d00a8f536bb69c5f9d6d2b9571e437c7ee5427299755ba1cecab56daeda6d8a  chunk4_long_t121_l16.safetensors

chunk4_short_t21_l16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fc4621c37451749b33d6b6493a7d2a6d3ffe5f0486c9631ed092ed990a8d9d9
+size 8236764356

chunk4_short_t21_l16.safetensors.sha256

@@ -0,0 +1 @@
+6fc4621c37451749b33d6b6493a7d2a6d3ffe5f0486c9631ed092ed990a8d9d9  chunk4_short_t21_l16.safetensors

chunk4_teacher_t21_l16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b240cb236aba1557c917193e2c31069a6808942cf2b7ddfc3bfb10520dd8217
+size 8236764356

chunk4_teacher_t21_l16.safetensors.sha256

@@ -0,0 +1 @@
+1b240cb236aba1557c917193e2c31069a6808942cf2b7ddfc3bfb10520dd8217  chunk4_teacher_t21_l16.safetensors

chunk4_teacher_t29_l24_camera.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68d9a0490db10c8c3d897b50e01143743ae494dd7ef902c676f4f005cfb5b278
+size 8589089124

chunk4_teacher_t29_l24_camera.safetensors.sha256

@@ -0,0 +1 @@
+68d9a0490db10c8c3d897b50e01143743ae494dd7ef902c676f4f005cfb5b278  chunk4_teacher_t29_l24_camera.safetensors