import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("JuliaML/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16", dtype=torch.bfloat16, device_map="cuda")
pipe.to("cuda")
prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)
output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")Cosmos3 Super Image-to-Video 4-Step — INT4 G64 BF16
Independent transformer-only quantization of
nvidia/Cosmos3-Super-Image2Video-4Step.
This is not an official NVIDIA release.
Base revision: 226cebfe8a8eef2ded13d561b0b236838d0f2d33.
Runtime and verification code:
gtrg55/cosmos3-quant-mlx-cuda.
Format
| Property | Value |
|---|---|
| Quantization | weight-only signed symmetric INT4 (W4A16) |
| Group size | 64 |
| Compute / activations | BF16 |
| BF16 transformer I/O | proj_in, proj_out |
| Quantized matrices | 896 |
| Transformer safetensors | 35.70 GiB |
| SVD / Hadamard / dynamic allocation | disabled |
| Sampler | original distilled 4-step schedule, guidance 1.0 |
The repository contains only the quantized transformer. The VAE, tokenizer and scheduler are loaded from the original NVIDIA model.
Validation
- Exact packed-weight sample round-trip: passed.
- Selected-tensor relative MSE:
0.013438441. - Apple Silicon MLX transformer + PyTorch/MPS VAE: passed end-to-end.
- CUDA SDNQ checkpoint format: supported by the companion loader; performance has not yet been benchmarked on NVIDIA hardware.
M4 Max smoke test at 512×512, 9 decoded frames, 8 exported frames, seed 1143:
| Metric | Result |
|---|---|
| Transformer load | 5.13 s |
| End-to-end generation | 80.89 s |
| MLX peak memory | 38.93 GiB |
Apple Silicon example
Clone the companion runtime, then download the original shared components and this quantized transformer:
hf download nvidia/Cosmos3-Super-Image2Video-4Step \
--revision 226cebfe8a8eef2ded13d561b0b236838d0f2d33 \
--exclude 'transformer/*' \
--local-dir models/Cosmos3-Super-Image2Video-4Step
hf download JuliaML/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16 \
--local-dir checkpoints/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16
PYTORCH_ENABLE_MPS_FALLBACK=1 python mlx/run_i2v_pipeline.py \
--model models/Cosmos3-Super-Image2Video-4Step \
--checkpoint checkpoints/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16/transformer \
--prompt checkpoints/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16/examples/prompt.json \
--image checkpoints/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16/examples/CookWhatLogo.png \
--latents checkpoints/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16/examples/cpu-seed1143-9f-512x512-bf16.safetensors \
--output outputs/cosmos3-int4.mp4 \
--height 512 --width 512 --num-frames 9 \
--export-frames 8 --sample-export-frames --condition-last-frame \
--fps 8 --seed 1143
The companion repository also contains cloud/run_sdnq_i2v_cuda.py for CUDA.
Reproducibility
The examples/ directory contains the exact prompt, conditioning image, fixed
BF16 noise tensor, run report, frame sheet, output MP4 and quantization
verification report.
- prompt SHA-256:
cb6be3182cc9171761159168e346d67770487f46858a377c81790a624dc61889 - latent SHA-256:
08d7b4cb9437944ca8c6e048426181b7459ae6030c83fe59099d9472f72d14e5 - input PNG SHA-256:
10a51ed3b6b0d9993157588893c12d8697bce457b534d393f47989ee6cb08d24 - output MP4 SHA-256:
99dd74b30057a73ed72c95027340cfbf57aed64198a8de389447a564edf49f09
Limitations
- The reference clip is a deliberately short functional test. NVIDIA recommends 480p and uses 189 frames by default; evaluate longer videos before production use.
- This is weight-only quantization, not integer activation inference.
- Quantization may alter motion, detail, prompt adherence and temporal stability.
- Safety behavior is inherited from the base model and runtime configuration.
License and attribution
Distributed under OpenMDW-1.1. See LICENSE and NOTICE. Preserve the NVIDIA
notices and license when redistributing these model materials.
- Downloads last month
- -
8-bit
Model tree for JuliaML/Cosmos3-Super-Image2Video-4Step-INT4-G64-BF16
Base model
nvidia/Cosmos3-Super-Image2Video-4Step