nielsr HF Staff commited on
Commit
7f98e19
·
verified ·
1 Parent(s): 3965cee

Add pipeline tag, library name, and sample usage

Browse files

This PR improves the model card for Cosmos3-Super by:
- Adding the `pipeline_tag: any-to-any` to reflect its omnimodal capabilities.
- Setting `library_name: diffusers` to enable the Hub's automated code snippets.
- Adding a sample usage section with a code snippet for the `diffusers` integration.
- Including a link to the paper page on Hugging Face.

Files changed (1) hide show
  1. README.md +45 -14
README.md CHANGED
@@ -1,22 +1,22 @@
1
  ---
 
2
  license: other
3
  license_name: openmdw1.1-license
4
- license_link: >-
5
- https://openmdw.ai/license/1-1/
6
- library_name: cosmos
7
  tags:
8
- - nvidia
9
- - cosmos
10
- - cosmos3
11
- - vllm
12
- - vllm-omni
13
- - diffusers
14
- - text, image, video, audio, and action generation
15
- - omnimodel
16
  ---
17
 
18
  # **Cosmos 3: Omnimodal World Models for Physical AI**
19
- **[Model Collection](https://huggingface.co/collections/nvidia/cosmos3)** | **[Code](https://github.com/nvidia/cosmos)** | **[White Paper](https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf)** | **[Website](https://research.nvidia.com/labs/cosmos-lab/cosmos3/)**
20
 
21
  [NVIDIA Cosmos™](https://github.com/nvidia/cosmos) is a world foundation model platform designed to accelerate the development of Physical AI by enabling machines to understand, simulate, and interact with the physical world across robotics, autonomous driving, and smart space environments, including industrial and factory-scale applications.
22
 
@@ -30,6 +30,37 @@ This model is ready for commercial and non-commercial use.
30
 
31
  **Model Developer:** NVIDIA
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ### Model Versions
34
  - Cosmos3-Nano:
35
  - Given multimodal inputs including text, images, video, audio, and action trajectories, generate coherent text, images, video, audio, and action outputs for multimodal understanding, world simulation, future prediction, action reasoning, and Physical AI applications.
@@ -816,7 +847,7 @@ client = openai.OpenAI(
816
  base_url="http://localhost:8000/v1",
817
  )
818
 
819
- response = client.chat.completions.create(
820
  model=client.models.list().data[0].id,
821
  messages=[
822
  {
@@ -941,4 +972,4 @@ Please make sure you have proper rights and permissions for all input image and
941
 
942
  Users are responsible for model inputs and outputs. Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms, prior to deployment.
943
 
944
- For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](EXPLAINABILITY.md), [Bias](BIAS.md), [Safety & Security](SAFETY.md), and [Privacy](PRIVACY.md) subcards. Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
 
1
  ---
2
+ library_name: diffusers
3
  license: other
4
  license_name: openmdw1.1-license
5
+ license_link: https://openmdw.ai/license/1-1/
6
+ pipeline_tag: any-to-any
 
7
  tags:
8
+ - nvidia
9
+ - cosmos
10
+ - cosmos3
11
+ - vllm
12
+ - vllm-omni
13
+ - diffusers
14
+ - text, image, video, audio, and action generation
15
+ - omnimodel
16
  ---
17
 
18
  # **Cosmos 3: Omnimodal World Models for Physical AI**
19
+ **[Paper Page](https://huggingface.co/papers/2606.02800)** | **[Model Collection](https://huggingface.co/collections/nvidia/cosmos3)** | **[Code](https://github.com/nvidia/cosmos)** | **[White Paper](https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf)** | **[Website](https://research.nvidia.com/labs/cosmos-lab/cosmos3/)**
20
 
21
  [NVIDIA Cosmos™](https://github.com/nvidia/cosmos) is a world foundation model platform designed to accelerate the development of Physical AI by enabling machines to understand, simulate, and interact with the physical world across robotics, autonomous driving, and smart space environments, including industrial and factory-scale applications.
22
 
 
30
 
31
  **Model Developer:** NVIDIA
32
 
33
+ ### Sample Usage
34
+
35
+ You can use the model with the [diffusers](https://github.com/huggingface/diffusers) library as shown below:
36
+
37
+ ```python
38
+ import torch
39
+ from diffusers import Cosmos3OmniPipeline
40
+ from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler
41
+ from diffusers.utils import export_to_video
42
+
43
+ pipe = Cosmos3OmniPipeline.from_pretrained(
44
+ "nvidia/Cosmos3-Super",
45
+ torch_dtype=torch.bfloat16,
46
+ device_map="cuda",
47
+ )
48
+ pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config, flow_shift=10.0)
49
+
50
+ result = pipe(
51
+ prompt="A mobile robot navigates a warehouse aisle and stops at a shelf.",
52
+ num_frames=189,
53
+ height=720,
54
+ width=1280,
55
+ fps=24,
56
+ num_inference_steps=35,
57
+ guidance_scale=6.0,
58
+ generator=torch.Generator(device="cuda").manual_seed(123),
59
+ )
60
+
61
+ export_to_video(result.video, "cosmos3_super_t2v.mp4", fps=24)
62
+ ```
63
+
64
  ### Model Versions
65
  - Cosmos3-Nano:
66
  - Given multimodal inputs including text, images, video, audio, and action trajectories, generate coherent text, images, video, audio, and action outputs for multimodal understanding, world simulation, future prediction, action reasoning, and Physical AI applications.
 
847
  base_url="http://localhost:8000/v1",
848
  )
849
 
850
+ response = client.chat.completion.create(
851
  model=client.models.list().data[0].id,
852
  messages=[
853
  {
 
972
 
973
  Users are responsible for model inputs and outputs. Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms, prior to deployment.
974
 
975
+ For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](EXPLAINABILITY.md), [Bias](BIAS.md), [Safety & Security](SAFETY.md), and [Privacy](PRIVACY.md) subcards. Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).