tangyue0820 qianlim commited on
Commit
fda1636
·
1 Parent(s): 226cebf

Refine model card overview (#21)

Browse files

- Refine I2V 4-step model card overview (98a91d4daae4d23d2dfee1116db8ecff55420fa1)


Co-authored-by: Qianli <qianlim@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -22,6 +22,16 @@ tags:
22
 
23
  ## Description
24
 
 
 
 
 
 
 
 
 
 
 
25
  Cosmos3 is a collection of Omnimodal world models capable of generating dynamic, high-quality video, image, audio, and action commands from combinations of text, image, video, and action trajectory inputs. It serves as a foundational building block for a broad range of Physical AI applications and research spanning world understanding, world generation, simulation, and embodied policy learning.
26
 
27
  This model is ready for commercial and non-commercial use.
@@ -333,12 +343,13 @@ Latency is reported in seconds. The suffix in each resolution column indicates t
333
  5. vLLM-Omni source results report "Avg. Generation Time" for the corresponding vLLM-Omni inference configuration.
334
  6. Cells marked `N/A` indicate configurations unavailable because the 64B model does not fit on a single GPU. Cells marked `Not tested` indicate configurations for which no source measurement was reported.
335
  7. All displayed values were calculated as `Cosmos3-Super source latency ÷ 17.5` and rounded to three decimal places.
 
336
 
337
  </sub>
338
 
339
  ## Usage: Prepare Prompts with Prompt Upsampling
340
 
341
- `Cosmos3-Super-Image2Video-4Step` supports dense natural-language prompts together with a first-frame image. It also supports JSON-format upsampled prompts, which can further improve generation quality. Description and code can be found [here](https://github.com/nvidia/cosmos-framework/blob/main/docs/prompt_upsampling.md).
342
 
343
  To create an upsampled prompt using Opus-4.7:
344
 
 
22
 
23
  ## Description
24
 
25
+ Cosmos3-Super-Image2Video-4Step is a 4-step distilled version of the base [Cosmos3-Super-Image2Video](https://huggingface.co/nvidia/Cosmos3-Super-Image2Video) model. Given an input image plus an optional text prompt, it generates a temporally coherent video.
26
+
27
+ - **What's different from the base model:** Distilled into a fixed 4-step sampling schedule without the need for classifier-free guidance. At the recommended 50-step + CFG setting for the Image2Video model, this replaces 50 × 2 = 100 model evaluations with 4, leading up to 25× fewer diffusion-model evaluations per video (see [PBR](https://huggingface.co/nvidia/Cosmos3-Super-Image2Video-4Step#pbr-performance-benchmark-reporting) for latency estimates).
28
+ - **Quality:** Ranked #1 among open-source models on the Artificial Analysis Image-to-Video leaderboard (as of 2026/07/17; see [Benchmarks section](https://huggingface.co/nvidia/Cosmos3-Super-Image2Video-4Step#benchmarks)).
29
+ - **Input / Output:** Image(s) in → MP4 video out. Recommended 480p (832×480, 16:9); 189 frames by default.
30
+ - **Example usage and output:** See [Usage](https://huggingface.co/nvidia/Cosmos3-Super-Image2Video-4Step#usage-run-inference-with-vllm-omni) section.
31
+ - **Hardware:** The 64B model needs a multi-GPU H100/H200 node (4–8 GPUs) or a single B200; it does not fit on a single smaller GPU. See [Usage](https://huggingface.co/nvidia/Cosmos3-Super-Image2Video-4Step#usage-run-inference-with-vllm-omni). Full model details below.
32
+
33
+ Cosmos3-Super-Image2Video-4Step was developed by NVIDIA as a part of Cosmos3.
34
+
35
  Cosmos3 is a collection of Omnimodal world models capable of generating dynamic, high-quality video, image, audio, and action commands from combinations of text, image, video, and action trajectory inputs. It serves as a foundational building block for a broad range of Physical AI applications and research spanning world understanding, world generation, simulation, and embodied policy learning.
36
 
37
  This model is ready for commercial and non-commercial use.
 
343
  5. vLLM-Omni source results report "Avg. Generation Time" for the corresponding vLLM-Omni inference configuration.
344
  6. Cells marked `N/A` indicate configurations unavailable because the 64B model does not fit on a single GPU. Cells marked `Not tested` indicate configurations for which no source measurement was reported.
345
  7. All displayed values were calculated as `Cosmos3-Super source latency ÷ 17.5` and rounded to three decimal places.
346
+ 8. Note on speedup basis: the 17.5× above is extrapolated from a 35-step base, which was the step count used in the original Cosmos3-Super base model speed benchmark. For Image-to-Video specifically, the recommended sampler setting is 50 steps, corresponding to `50 × 2 ÷ 4 = 25×` fewer model evaluations; this is the figure used in general reporting. The 17.5× estimate is retained here as the more conservative, benchmark-matched number.
347
 
348
  </sub>
349
 
350
  ## Usage: Prepare Prompts with Prompt Upsampling
351
 
352
+ `Cosmos3-Super-Image2Video-4Step` supports dense natural-language prompts together with a first-frame image. Prompt upsampling is optional for distilled models: you can run inference directly with a natural-language prompt, or use a JSON-format upsampled prompt for potential quality improvements. Description and code can be found [here](https://github.com/nvidia/cosmos-framework/blob/main/docs/prompt_upsampling.md).
353
 
354
  To create an upsampled prompt using Opus-4.7:
355