Instructions to use nvidia/Cosmos3-Edge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use nvidia/Cosmos3-Edge with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add image-to-video with negative prompt example
Browse filesAdds an 'Image to video with negative prompt' usage example that mirrors the existing image-to-video section, reusing the same input image and prompt but passing the curated assets/negative_prompt.json as the per-sample negative prompt (JSON-encoded string, matching the model's default negative caption format). Includes the generated example output (assets/edge_i2v_negprompt_output.mp4, 832x480, 121 frames @ 24fps).
- .gitattributes +1 -0
- README.md +43 -0
- assets/edge_i2v_negprompt_output.mp4 +3 -0
.gitattributes
CHANGED
|
@@ -51,3 +51,4 @@ assets/edge_i2v_output.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
| 51 |
assets/edge_action_fd_umi_2chunk_output.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 52 |
assets/edge_action_id_av_0_output.png filter=lfs diff=lfs merge=lfs -text
|
| 53 |
assets/edge_action_id_av_1_output.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 51 |
assets/edge_action_fd_umi_2chunk_output.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 52 |
assets/edge_action_id_av_0_output.png filter=lfs diff=lfs merge=lfs -text
|
| 53 |
assets/edge_action_id_av_1_output.png filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
assets/edge_i2v_negprompt_output.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -514,6 +514,49 @@ Example output:
|
|
| 514 |
|
| 515 |
<video controls width="832" height="480" src="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets/edge_i2v_output.mp4"></video>
|
| 516 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 517 |
#### Action — forward dynamics
|
| 518 |
|
| 519 |
Forward dynamics rolls out one action chunk at a time. This self-contained script fetches the UMI conditioning frame and action chunks, runs `cosmos_framework.scripts.inference` per chunk, feeds each chunk's final generated frame in as the next chunk's conditioning image, and stitches the 2-chunk rollout. Install `imageio[ffmpeg]` first.
|
|
|
|
| 514 |
|
| 515 |
<video controls width="832" height="480" src="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets/edge_i2v_output.mp4"></video>
|
| 516 |
|
| 517 |
+
#### Image to video with negative prompt
|
| 518 |
+
|
| 519 |
+
The same image-to-video setup as above, but with a structured **negative prompt** to steer the model away from common failure modes (temporal flicker, morphing geometry, broken physics, unstable camera push-in). We reuse the same input image and prompt, and pass the curated `negative_prompt.json` from `assets/` as the per-sample `negative_prompt` (JSON-encoded as a string, mirroring the model's default negative caption format).
|
| 520 |
+
|
| 521 |
+
```shell
|
| 522 |
+
BASE="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets"
|
| 523 |
+
python - "$BASE" <<'PY'
|
| 524 |
+
import json, sys, urllib.request
|
| 525 |
+
base = sys.argv[1]
|
| 526 |
+
prompt = urllib.request.urlopen(f"{base}/example_i2v_prompt.json").read().decode()
|
| 527 |
+
negative = urllib.request.urlopen(f"{base}/negative_prompt.json").read().decode()
|
| 528 |
+
json.dump(
|
| 529 |
+
{
|
| 530 |
+
"model_mode": "image2video",
|
| 531 |
+
"prompt": prompt,
|
| 532 |
+
"vision_path": f"{base}/example_i2v_input.jpg",
|
| 533 |
+
"negative_prompt": json.dumps(json.loads(negative)),
|
| 534 |
+
},
|
| 535 |
+
open("i2v_edge_neg.json", "w"), indent=2,
|
| 536 |
+
)
|
| 537 |
+
PY
|
| 538 |
+
|
| 539 |
+
python -m cosmos_framework.scripts.inference \
|
| 540 |
+
--parallelism-preset=latency \
|
| 541 |
+
-i i2v_edge_neg.json \
|
| 542 |
+
-o outputs/omni_edge_neg \
|
| 543 |
+
--checkpoint-path Cosmos3-Edge \
|
| 544 |
+
--resolution 480 \
|
| 545 |
+
--num-frames 121 \
|
| 546 |
+
--fps 24 \
|
| 547 |
+
--num-steps 50 \
|
| 548 |
+
--guidance 5.0 \
|
| 549 |
+
--shift 3.0 \
|
| 550 |
+
--sampler unipc \
|
| 551 |
+
--seed 0
|
| 552 |
+
```
|
| 553 |
+
|
| 554 |
+
The generated video is written to `outputs/omni_edge_neg/<sample_name>/vision.mp4`.
|
| 555 |
+
|
| 556 |
+
Example output:
|
| 557 |
+
|
| 558 |
+
<video controls width="832" height="480" src="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets/edge_i2v_negprompt_output.mp4"></video>
|
| 559 |
+
|
| 560 |
#### Action — forward dynamics
|
| 561 |
|
| 562 |
Forward dynamics rolls out one action chunk at a time. This self-contained script fetches the UMI conditioning frame and action chunks, runs `cosmos_framework.scripts.inference` per chunk, feeds each chunk's final generated frame in as the next chunk's conditioning image, and stitches the 2-chunk rollout. Install `imageio[ffmpeg]` first.
|
assets/edge_i2v_negprompt_output.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f2d41ae997adea1eef0f046b837404d6add2798e050974ea36696cee4b3efd00
|
| 3 |
+
size 23017533
|