liang1225 commited on
Commit
070d251
·
verified ·
1 Parent(s): 0fae1e9

Make image-to-video example use the negative prompt by default

Browse files

Replaces the plain 'Image to video' example with the negative-prompt variant so there is a single image-to-video section that passes the curated assets/negative_prompt.json as the per-sample negative prompt (JSON-encoded string). Drops the separate 'Image to video with negative prompt' section and points the example output at assets/edge_i2v_negprompt_output.mp4.

Files changed (1) hide show
  1. README.md +5 -40
README.md CHANGED
@@ -481,42 +481,7 @@ Cosmos3-Edge is a compact omni model that fits on a single GPU and is the recomm
481
 
482
  #### Image to video
483
 
484
- ```shell
485
- BASE="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets"
486
- python - "$BASE" <<'PY'
487
- import json, sys, urllib.request
488
- base = sys.argv[1]
489
- prompt = urllib.request.urlopen(f"{base}/example_i2v_prompt.json").read().decode()
490
- json.dump(
491
- {"model_mode": "image2video", "prompt": prompt, "vision_path": f"{base}/example_i2v_input.jpg"},
492
- open("i2v_edge.json", "w"), indent=2,
493
- )
494
- PY
495
-
496
- python -m cosmos_framework.scripts.inference \
497
- --parallelism-preset=latency \
498
- -i i2v_edge.json \
499
- -o outputs/omni_edge \
500
- --checkpoint-path Cosmos3-Edge \
501
- --resolution 480 \
502
- --num-frames 121 \
503
- --fps 24 \
504
- --num-steps 50 \
505
- --guidance 5.0 \
506
- --shift 3.0 \
507
- --sampler unipc \
508
- --seed 0
509
- ```
510
-
511
- The generated video is written to `outputs/omni_edge/<sample_name>/vision.mp4`.
512
-
513
- Example output:
514
-
515
- <video controls width="832" height="480" src="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets/edge_i2v_output.mp4"></video>
516
-
517
- #### Image to video with negative prompt
518
-
519
- The same image-to-video setup as above, but with a structured **negative prompt** to steer the model away from common failure modes (temporal flicker, morphing geometry, broken physics, unstable camera push-in). We reuse the same input image and prompt, and pass the curated `negative_prompt.json` from `assets/` as the per-sample `negative_prompt` (JSON-encoded as a string, mirroring the model's default negative caption format).
520
 
521
  ```shell
522
  BASE="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets"
@@ -532,14 +497,14 @@ json.dump(
532
  "vision_path": f"{base}/example_i2v_input.jpg",
533
  "negative_prompt": json.dumps(json.loads(negative)),
534
  },
535
- open("i2v_edge_neg.json", "w"), indent=2,
536
  )
537
  PY
538
 
539
  python -m cosmos_framework.scripts.inference \
540
  --parallelism-preset=latency \
541
- -i i2v_edge_neg.json \
542
- -o outputs/omni_edge_neg \
543
  --checkpoint-path Cosmos3-Edge \
544
  --resolution 480 \
545
  --num-frames 121 \
@@ -551,7 +516,7 @@ python -m cosmos_framework.scripts.inference \
551
  --seed 0
552
  ```
553
 
554
- The generated video is written to `outputs/omni_edge_neg/<sample_name>/vision.mp4`.
555
 
556
  Example output:
557
 
 
481
 
482
  #### Image to video
483
 
484
+ Image-to-video generation steered by a structured **negative prompt** that keeps the model away from common failure modes (temporal flicker, morphing geometry, broken physics, unstable camera push-in). We pass the curated `negative_prompt.json` from `assets/` as the per-sample `negative_prompt` (JSON-encoded as a string, mirroring the model's default negative caption format).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
485
 
486
  ```shell
487
  BASE="https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets"
 
497
  "vision_path": f"{base}/example_i2v_input.jpg",
498
  "negative_prompt": json.dumps(json.loads(negative)),
499
  },
500
+ open("i2v_edge.json", "w"), indent=2,
501
  )
502
  PY
503
 
504
  python -m cosmos_framework.scripts.inference \
505
  --parallelism-preset=latency \
506
+ -i i2v_edge.json \
507
+ -o outputs/omni_edge \
508
  --checkpoint-path Cosmos3-Edge \
509
  --resolution 480 \
510
  --num-frames 121 \
 
516
  --seed 0
517
  ```
518
 
519
+ The generated video is written to `outputs/omni_edge/<sample_name>/vision.mp4`.
520
 
521
  Example output:
522