Instructions to use nvidia/Cosmos3-Super-Image2Video-4Step with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use nvidia/Cosmos3-Super-Image2Video-4Step with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
List Diffusers as a runtime engine and trim the Diffusers section
Browse files
README.md
CHANGED
|
@@ -493,9 +493,7 @@ Example output generated from `assets/example_first_frame.png`:
|
|
| 493 |
|
| 494 |
## Usage: Run Inference with Diffusers
|
| 495 |
|
| 496 |
-
`Cosmos3-Super-Image2Video-4Step` is supported by the Hugging Face Diffusers **modular** pipeline
|
| 497 |
-
|
| 498 |
-
This path loads the full 64B model on a single device in BF16, so it needs a B200/GB200-class GPU. On H100/H200-class GPUs, use the multi-GPU vLLM-Omni serving configuration above instead.
|
| 499 |
|
| 500 |
### Install
|
| 501 |
|
|
@@ -521,8 +519,6 @@ The safety checker requires access to the gated [nvidia/Cosmos-1.0-Guardrail](ht
|
|
| 521 |
|
| 522 |
### Example: Image to Video Generation
|
| 523 |
|
| 524 |
-
Run this from the downloaded repo root, using the same `assets/example_prompt.json` and `assets/example_first_frame.png` inputs as the vLLM-Omni request. The example generates at 480p (`832x480` at 16:9), the recommended setting for this distilled I2V checkpoint.
|
| 525 |
-
|
| 526 |
```python
|
| 527 |
import json
|
| 528 |
|
|
@@ -530,9 +526,6 @@ import torch
|
|
| 530 |
from diffusers import Cosmos3DistilledModularPipeline
|
| 531 |
from diffusers.utils import export_to_video, load_image
|
| 532 |
|
| 533 |
-
# The prompt file holds the same positive and negative prompt fields the vLLM-Omni
|
| 534 |
-
# example uses. Only the positive prompt is passed here: classifier-free guidance is
|
| 535 |
-
# baked into the distilled weights, so the negative prompt has no effect.
|
| 536 |
json_prompt = json.load(open("assets/example_prompt.json"))
|
| 537 |
image = load_image("assets/example_first_frame.png")
|
| 538 |
|
|
|
|
| 493 |
|
| 494 |
## Usage: Run Inference with Diffusers
|
| 495 |
|
| 496 |
+
`Cosmos3-Super-Image2Video-4Step` is supported by the Hugging Face Diffusers **modular** pipeline. The distilled checkpoint is not compatible with `Cosmos3OmniPipeline` or `Cosmos3OmniModularPipeline`; it must be loaded with `Cosmos3DistilledModularPipeline`.
|
|
|
|
|
|
|
| 497 |
|
| 498 |
### Install
|
| 499 |
|
|
|
|
| 519 |
|
| 520 |
### Example: Image to Video Generation
|
| 521 |
|
|
|
|
|
|
|
| 522 |
```python
|
| 523 |
import json
|
| 524 |
|
|
|
|
| 526 |
from diffusers import Cosmos3DistilledModularPipeline
|
| 527 |
from diffusers.utils import export_to_video, load_image
|
| 528 |
|
|
|
|
|
|
|
|
|
|
| 529 |
json_prompt = json.load(open("assets/example_prompt.json"))
|
| 530 |
image = load_image("assets/example_first_frame.png")
|
| 531 |
|