Instructions to use wfen/Cosmos3-Nano-NVFP4-Blockwise with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use wfen/Cosmos3-Nano-NVFP4-Blockwise with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("wfen/Cosmos3-Nano-NVFP4-Blockwise", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
initialize Blockwise NVFP4 quantization model
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +8 -0
- BIAS.md +3 -0
- EXPLAINABILITY.md +3 -0
- PRIVACY.md +3 -0
- README.md +3 -3
- SAFETY.md +3 -0
- assets/example_action_fd_agibotworld_action_chunks.json +3 -0
- assets/example_action_id_av_0_output.json +3 -0
- assets/example_action_id_av_1_output.json +3 -0
- assets/example_i2v_input.jpg +3 -0
- assets/example_i2v_prompt.json +3 -0
- assets/example_reasoning_prompt.json +3 -0
- assets/example_t2v_prompt.json +3 -0
- assets/example_t2v_prompt_short.txt +3 -0
- assets/example_t2vs_prompt.json +3 -0
- assets/negative_prompt.json +3 -0
- assets/nvfp4-forward_dynamics.mp4 +3 -0
- assets/nvfp4-i2v.mp4 +3 -0
- assets/nvfp4-t2v-flagship.mp4 +3 -0
- assets/nvfp4-t2v_audio.mp4 +3 -0
- assets/nvfp4_t2i.png +3 -0
- assets/nvfp4_t2v_baseline.mp4 +3 -0
- assets/nvfp4_t2v_fast.mp4 +3 -0
- chat_template.json +3 -0
- checkpoint.json +3 -0
- config.json +3 -0
- generation_config.json +3 -0
- images/benchmark-action-1.png +3 -0
- images/benchmark-overall.png +3 -0
- images/benchmark-reasoning.png +3 -0
- images/benchmark-visual-audio.png +3 -0
- merges.txt +3 -0
- model.safetensors.index.json +3 -0
- model_index.json +3 -0
- preprocessor_config.json +3 -0
- scheduler/scheduler_config.json +3 -0
- sound_tokenizer/config.json +3 -0
- sound_tokenizer/diffusion_pytorch_model.safetensors +3 -0
- text_tokenizer/added_tokens.json +3 -0
- text_tokenizer/chat_template.jinja +3 -0
- text_tokenizer/merges.txt +3 -0
- text_tokenizer/special_tokens_map.json +3 -0
- text_tokenizer/tokenizer.json +3 -0
- text_tokenizer/tokenizer_config.json +3 -0
- text_tokenizer/vocab.json +3 -0
- tokenizer.json +3 -0
- tokenizer_config.json +3 -0
- transformer/config.json +3 -0
- transformer/model.safetensors +3 -0
- transformer/nvfp4_blockwise_mixed_v1.json +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
text_tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
*.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
*.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
*.md filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
*.json filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
*.jpg filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
*.txt filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
*.jinja filter=lfs diff=lfs merge=lfs -text
|
BIAS.md
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1d65943c20dac47c4250b5a4a71266c001f6c85a17dad556ad00dbfc2a8115a9
|
| 3 |
+
size 4720
|
EXPLAINABILITY.md
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7d9e73df359fc9c0fcffdc18045018f55f124e5a6b6f71268a93d63c56e1a997
|
| 3 |
+
size 3189
|
PRIVACY.md
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0298f236e550d4064cd8d83babad8dfc791295bba602210ca6c8106a993ca699
|
| 3 |
+
size 1215
|
README.md
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
-
-
|
| 2 |
-
|
| 3 |
-
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:656f1a410d6be6319f09693590acb1df7acdbe442de90cbc5496ae92e66c5f3b
|
| 3 |
+
size 43813
|
SAFETY.md
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b59cf989cb66eb3b1c5e0692d04c42b9a39751801259b225733ab9493e08f03b
|
| 3 |
+
size 3677
|
assets/example_action_fd_agibotworld_action_chunks.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ba8408f727f9c77d4450b239069f81e9cdd9d099bf05da7f33f5bfb4cb2d55cd
|
| 3 |
+
size 48386
|
assets/example_action_id_av_0_output.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0c6a6fe8f1bddd1e119a37e66a168b82ef62e4ee99c024370d2d25c80ee83a24
|
| 3 |
+
size 15976
|
assets/example_action_id_av_1_output.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5ee587d25fb6e5fe5f54febaa63d7890c029b141f2eed047b1b7ee93877fde3e
|
| 3 |
+
size 15704
|
assets/example_i2v_input.jpg
ADDED
|
Git LFS Details
|
assets/example_i2v_prompt.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:baa77ef7ae92c889a96ff0402db283ff07d9da881d31abfa8b986f8481cb8273
|
| 3 |
+
size 9176
|
assets/example_reasoning_prompt.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fb6fbd57e5af6b09960d9960c709e1b422faf299a4b34139d212c7fccc488348
|
| 3 |
+
size 151
|
assets/example_t2v_prompt.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:61c9c4b46b6787d967cc509a2bf323766e70bf5ecf40e09a739362beac135677
|
| 3 |
+
size 7710
|
assets/example_t2v_prompt_short.txt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cc739831cb4ce9c8de022522c0486943e7c5565978d6fae4f99836be2c380e73
|
| 3 |
+
size 46
|
assets/example_t2vs_prompt.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bb23395fe07806a1af6d1dd57299816924b79f306bc789a05f1fb1176f0ecf67
|
| 3 |
+
size 8885
|
assets/negative_prompt.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7a60bd0c1335641cddd8b3ae66de767d2282f08861324f0ebe850e613af6a3a1
|
| 3 |
+
size 15379
|
assets/nvfp4-forward_dynamics.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:49820c25767322716bbf41ca4468b98939c31d3e17607dba1c16f8107773769d
|
| 3 |
+
size 628708
|
assets/nvfp4-i2v.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ad10e92e96405d3337b8c0ace9afa8a5c1cc13b1c8813c9716980ee06457231a
|
| 3 |
+
size 2354183
|
assets/nvfp4-t2v-flagship.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1d4c011990253d1ee94b8f14cf675c1eb80bc789a45531009a6615a69a02133b
|
| 3 |
+
size 9086281
|
assets/nvfp4-t2v_audio.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:58c61f8d34366b214b501fb3da3e9af35ca822009de772c681eb6f6ba85dd672
|
| 3 |
+
size 711869
|
assets/nvfp4_t2i.png
ADDED
|
Git LFS Details
|
assets/nvfp4_t2v_baseline.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4e60b4fb2e6024796693538b2ddff00138ed8564abbbcf2af936f22ea4663dd3
|
| 3 |
+
size 3806724
|
assets/nvfp4_t2v_fast.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e01fe7dae81acccb72f96e92c894b4b4d8e2544617b2a83cf83c60acfb9cc524
|
| 3 |
+
size 1024488
|
chat_template.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5c72a170d2a4a1a3bc5adad2e689ae28138a9700e5b8c96c0266331e86c0acce
|
| 3 |
+
size 5499
|
checkpoint.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a
|
| 3 |
+
size 2
|
config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c32f2468a54542c21946bc8eab6172b911dcec9a7193a94c023ea2d4073bcda6
|
| 3 |
+
size 8166
|
generation_config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8469742d1fce0de951c8909b26a2c0c0d8490837ce476efb114da9e0cefc4d44
|
| 3 |
+
size 269
|
images/benchmark-action-1.png
ADDED
|
Git LFS Details
|
images/benchmark-overall.png
ADDED
|
Git LFS Details
|
images/benchmark-reasoning.png
ADDED
|
Git LFS Details
|
images/benchmark-visual-audio.png
ADDED
|
Git LFS Details
|
merges.txt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:599bab54075088774b1733fde865d5bd747cbcc7a547c5bc12610e874e26f5e3
|
| 3 |
+
size 1671839
|
model.safetensors.index.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:43bdea0b60472d7b4f74f280d5eb508ec5e895f24287af8d2ecf2d0e2738708f
|
| 3 |
+
size 112969
|
model_index.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:90088a6638deb68418d795971ad48e6d5aa3f7764d69f3c62bbca9e8f3485bcd
|
| 3 |
+
size 502
|
preprocessor_config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516
|
| 3 |
+
size 390
|
scheduler/scheduler_config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3e25a6a5b472e77ba45cb17a4956e70ff922b996e4f9bf421f56e7b19d1359ec
|
| 3 |
+
size 884
|
sound_tokenizer/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a8194162a0126753981aa81054ab14382f01776fcc56e96cde4867a87bf4e8da
|
| 3 |
+
size 1310
|
sound_tokenizer/diffusion_pytorch_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9d4c61cde38acfb0cad9048a140c3533750277a8462b19dc08450d9fe1ad9879
|
| 3 |
+
size 1892409600
|
text_tokenizer/added_tokens.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c0284b582e14987fbd3d5a2cb2bd139084371ed9acbae488829a1c900833c680
|
| 3 |
+
size 707
|
text_tokenizer/chat_template.jinja
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3636d0f0bd6bef02654cdffdc447b79cb2cef8ab02cc75267345946291a489e4
|
| 3 |
+
size 5292
|
text_tokenizer/merges.txt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5
|
| 3 |
+
size 1671853
|
text_tokenizer/special_tokens_map.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:76862e765266b85aa9459767e33cbaf13970f327a0e88d1c65846c2ddd3a1ecd
|
| 3 |
+
size 613
|
text_tokenizer/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
|
| 3 |
+
size 11422654
|
text_tokenizer/tokenizer_config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:355f2b4e5bad7b01f11ef6cb68ebc176f61b95c3276092ea225b1bea0e01e95c
|
| 3 |
+
size 5404
|
text_tokenizer/vocab.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910
|
| 3 |
+
size 2776833
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a5d85b6dcc535e6b93115a9ef287e6132fdbf30270da6218194ba742261173c7
|
| 3 |
+
size 7032403
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c2da771801886ad9ae98181793ffd3dfb7f1af30f6f7c6a4e15d7dbba52e2399
|
| 3 |
+
size 10868
|
transformer/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d1ed91f3c34d5901e8c7ab292622d2380bc6455b05c61d917db2c1795dd09cab
|
| 3 |
+
size 1453
|
transformer/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8c3e1aa3360a867410e78d2b8c922f4732ebc9ebc8bd8bdb5f95e1297f8026cb
|
| 3 |
+
size 14719504288
|
transformer/nvfp4_blockwise_mixed_v1.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:98485b929c744ae53903d0e02c1e5977d9ddc6a757b48da1608e8a8edc5e530b
|
| 3 |
+
size 51434
|