Image-to-Video
Cosmos
Diffusers
Safetensors
cosmos3_omni
nvidia
cosmos3
vllm-omni
sglang
sglang-diffusion
video-generation
Instructions to use nvidia/Cosmos3-Super-Image2Video with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use nvidia/Cosmos3-Super-Image2Video with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Diffusers
How to use nvidia/Cosmos3-Super-Image2Video with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("nvidia/Cosmos3-Super-Image2Video", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Commit ·
18c74df
1
Parent(s): b2a1b0d
add sglang to more sections
Browse files
README.md
CHANGED
|
@@ -168,6 +168,7 @@ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated sys
|
|
| 168 |
- [PyTorch](https://github.com/nvidia/cosmos3)
|
| 169 |
- [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
|
| 170 |
- [Hugging Face Diffusers](https://huggingface.co/docs/diffusers/en/index)
|
|
|
|
| 171 |
|
| 172 |
**Supported Hardware Microarchitecture Compatibility:**
|
| 173 |
|
|
@@ -416,7 +417,7 @@ python scripts/upsample_prompt.py \
|
|
| 416 |
|
| 417 |
### SGLang
|
| 418 |
|
| 419 |
-
SGLang
|
| 420 |
|
| 421 |
```bash
|
| 422 |
git clone --branch main https://github.com/sgl-project/sglang.git
|
|
@@ -460,6 +461,8 @@ curl -sS -L "http://localhost:30000/v1/videos/${job_id}/content" \
|
|
| 460 |
-o cosmos3_super_i2v_output.mp4
|
| 461 |
```
|
| 462 |
|
|
|
|
|
|
|
| 463 |
### Diffusers
|
| 464 |
|
| 465 |
Cosmos3 is fully supported within the popular HuggingFace Diffusers package. This integration makes it a supported inference backend, allowing developers to easily incorporate Cosmos3's capabilities - such as text-to-image generation - into their pipelines using the Cosmos3OmniPipeline class, as demonstrated by the provided code examples (see examples for other modalities on the HuggingFace Cosmos3 page).
|
|
@@ -538,7 +541,7 @@ Cosmos3 outputs should not be treated as physically accurate simulation, reliabl
|
|
| 538 |
|
| 539 |
## Inference
|
| 540 |
|
| 541 |
-
**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM](https://github.com/vllm-project/vllm), [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [Hugging Face Diffusers](https://github.com/huggingface/diffusers)
|
| 542 |
|
| 543 |
**Test Hardware:** GB200 and H100
|
| 544 |
|
|
|
|
| 168 |
- [PyTorch](https://github.com/nvidia/cosmos3)
|
| 169 |
- [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
|
| 170 |
- [Hugging Face Diffusers](https://huggingface.co/docs/diffusers/en/index)
|
| 171 |
+
- [SGLang](https://sgl-project.github.io/)
|
| 172 |
|
| 173 |
**Supported Hardware Microarchitecture Compatibility:**
|
| 174 |
|
|
|
|
| 417 |
|
| 418 |
### SGLang
|
| 419 |
|
| 420 |
+
[SGLang Diffusion](https://sgl-project.github.io/diffusion) can serve `nvidia/Cosmos3-Super-Image2Video` through OpenAI-compatible video generation endpoints. Install SGLang from the main branch with diffusion dependencies, then start the server:
|
| 421 |
|
| 422 |
```bash
|
| 423 |
git clone --branch main https://github.com/sgl-project/sglang.git
|
|
|
|
| 461 |
-o cosmos3_super_i2v_output.mp4
|
| 462 |
```
|
| 463 |
|
| 464 |
+
For complete serving instructions and request examples, see the [Cosmos3 SGLang cookbook](https://lmsysorg.mintlify.app/cookbook/diffusion/Cosmos/Cosmos3).
|
| 465 |
+
|
| 466 |
### Diffusers
|
| 467 |
|
| 468 |
Cosmos3 is fully supported within the popular HuggingFace Diffusers package. This integration makes it a supported inference backend, allowing developers to easily incorporate Cosmos3's capabilities - such as text-to-image generation - into their pipelines using the Cosmos3OmniPipeline class, as demonstrated by the provided code examples (see examples for other modalities on the HuggingFace Cosmos3 page).
|
|
|
|
| 541 |
|
| 542 |
## Inference
|
| 543 |
|
| 544 |
+
**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM](https://github.com/vllm-project/vllm), [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [Hugging Face Diffusers](https://github.com/huggingface/diffusers), [SGLang](https://sgl-project.github.io/), [SGLang Diffusion](https://sgl-project.github.io/diffusion)
|
| 545 |
|
| 546 |
**Test Hardware:** GB200 and H100
|
| 547 |
|