Release Cosmos3-Nano S1/S3 checkpoints and model card

README.md

@@ -7,14 +7,15 @@ library_name: artifixer
 ## Description:
  
 ArtiFixer is a few-step causal auto-regressive model that enhances and extends 3D reconstruction. The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and Self-Forcing-style DMD distillation.  
-ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and is released in two variants: a 14B variant based on Wan2.1's 14B model, and a lighter-weight 1.3B variant based on Wan2.1's 1.3B model that trades some reconstruction quality for significantly higher throughput and lower training and inference compute.  
-In addition to the distilled few-step causal auto-regressive models, this repository also provides the bidirectional diffusion **teacher** checkpoints from which each released model was distilled (`artifixer-s1-14b.pt` and `artifixer-s1-1.3b.pt`, the stage-1 bidirectional training outputs). The teachers use full bidirectional attention with standard multi-step diffusion sampling instead of few-step causal auto-regressive rollout. They are provided for research on distillation, sampling, and long-rollout behavior, and as the starting point for reproducing the diffusion-forcing and DMD distillation stages.  
+ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and provides distilled causal models and their bidirectional S1 teachers in three variants: Wan2.1 14B, Wan2.1 1.3B, and Cosmos3-Nano. The Wan2.1 1.3B variant trades some reconstruction quality for higher throughput and lower compute.  
+The distilled stage-3 (S3) checkpoints are [`artifixer-14b.pt`](./artifixer-14b.pt), [`artifixer-1.3b.pt`](./artifixer-1.3b.pt), and [`artifixer-cosmos3-nano.pt`](./artifixer-cosmos3-nano.pt).
+In addition to the distilled few-step causal auto-regressive models, this repository also provides the bidirectional diffusion **teacher** checkpoints from which each released model was distilled ([`artifixer-s1-14b.pt`](./artifixer-s1-14b.pt), [`artifixer-s1-1.3b.pt`](./artifixer-s1-1.3b.pt), and [`artifixer-s1-cosmos3-nano.pt`](./artifixer-s1-cosmos3-nano.pt), the stage-1 bidirectional training outputs). The teachers use full bidirectional attention with standard multi-step diffusion sampling instead of few-step causal auto-regressive rollout. They are provided for research on distillation, sampling, and long-rollout behavior, and as the starting point for reproducing the diffusion-forcing and DMD distillation stages.  
 _This model is for research and development only._
  
 ### License/Terms of Use:
  
 GOVERNING DOWNLOAD TERMS: Use of the model is governed by the [NVIDIA License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=).  
-ADDITIONAL INFORMATION: The Wan 2.1 14B and 1.3B base models are governed by the [Apache License, Version 2.0.](https://www.apache.org/licenses/LICENSE-2.0)
+ADDITIONAL INFORMATION: The Wan 2.1 14B and 1.3B base models are governed by the [Apache License, Version 2.0.](https://www.apache.org/licenses/LICENSE-2.0) The [Cosmos3-Nano](https://huggingface.co/nvidia/Cosmos3-Nano) base model is governed by the [Open MDW 1.1 License](https://openmdw.ai/license/1-1/).
  
 ### Deployment Geography:
  
@@ -26,7 +27,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
  
 ### Release Date:
  
-**Other:** Hugging Face: 06/04/2026 (14B variant), 07/21/2026 (1.3B variant), and 09/2026 (bidirectional teacher checkpoints) via https://research.nvidia.com/labs/sil/projects/artifixer/
+**Other:** Hugging Face: 06/04/2026 (14B variant), 07/21/2026 (1.3B variant), and 09/2026 (Wan bidirectional teacher checkpoints) via https://research.nvidia.com/labs/sil/projects/artifixer/
  
 ## Reference(s):
  
@@ -35,10 +36,18 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
 ## Model Architecture:
  
 **Architecture Type:** Transformer  
-**Network Architecture:** ArtifixerTransformer (built on Wan2.1's WanTransformer3DModel)  
-**This model was developed based on Wan-AI/Wan2.1-T2V-14B-Diffusers (14B variant) and Wan-AI/Wan2.1-T2V-1.3B-Diffusers (1.3B variant).**  
-**Number of model parameters:** 14B variant: ~16.9B trainable (16,910,955,584); 1.3B variant: ~1.68B trainable (1,678,749,760). The teacher checkpoints share the architecture and parameter counts of their respective variants.
- 
+**Network Architecture:** ArtifixerTransformer (built on Wan2.1's WanTransformer3DModel) and ArtifixerCosmos3Transformer (built on Cosmos3OmniTransformer)  
+**The variants are based on Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.1-T2V-1.3B-Diffusers, and nvidia/Cosmos3-Nano, respectively.**  
+**Number of model parameters:** 14B variant: ~16.9B trainable (16,910,955,584); 1.3B variant: ~1.68B trainable (1,678,749,760); Cosmos3-Nano: ~15.8B (15,802,891,556). The teacher checkpoints share the architecture and parameter counts of their respective variants.
+ 
+**Cosmos3-Nano architecture:** ArtiFixer's Cosmos3 transformer is based on
+`nvidia/Cosmos3-Nano` at revision `23314034b82f46b45339035dba67c3ee9bbcb8ba`.
+It uses token-ID prompts and
+SCoPE camera conditioning, with reference and target images in shared
+self-attention. The exported checkpoints contain the transformer weights for
+image-sequence reconstruction. The VAE, tokenizer, and configuration are loaded
+separately from the pinned base-model revision.
+
 ## Input:
  
 **Input Type(s):** Image, text  
@@ -51,7 +60,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
 **Output Type(s):** Image  
 **Output Format:** RGB (Red, Green, Blue)  
 **Output Parameters:** Two-Dimensional (2D) image frames  
-**Other Properties Related to Output:** Generates enhanced images via a few-step causal auto-regressive diffusion model. At 4 denoising steps on a single GPU, the 14B variant reaches 8.36 FPS and the 1.3B variant 34.38 FPS; with 1 denoising step and 4-GPU context parallelism, the 1.3B variant reaches up to 101.77 FPS. The bidirectional teacher checkpoints instead run standard multi-step diffusion sampling with full bidirectional attention; the FPS figures above apply to the distilled causal auto-regressive models only.
+**Other Properties Related to Output:** Generates enhanced images via a few-step causal auto-regressive diffusion model. At 4 denoising steps on a single GPU, the 14B variant reaches 8.36 FPS and the 1.3B variant 34.38 FPS; with 1 denoising step and 4-GPU context parallelism, the 1.3B variant reaches up to 101.77 FPS. These speed figures describe the Wan variants. The bidirectional teacher checkpoints instead run standard multi-step diffusion sampling with full bidirectional attention; the FPS figures above apply to the distilled causal auto-regressive models only.
  
 Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
  
@@ -65,7 +74,7 @@ The integration of foundation and fine-tuned models into AI systems requires add
  
 ## Model Version(s):
  
-ArtiFixer 14B v1.0, ArtiFixer 1.3B v1.0, ArtiFixer 14B Teacher v1.0, ArtiFixer 1.3B Teacher v1.0  
+ArtiFixer 14B v1.0, ArtiFixer 1.3B v1.0, ArtiFixer 14B Teacher v1.0, ArtiFixer 1.3B Teacher v1.0, ArtiFixer Cosmos3-Nano v1.0, ArtiFixer Cosmos3-Nano Teacher v1.0  
 The teacher checkpoints are the stage-1 bidirectional models from which the corresponding released causal auto-regressive models were distilled.  
 ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Dockerfiles for CUDA 12 and 13, supporting both x86_64 and aarch64 architectures. The model can be run using torchrun (or accelerate) with multi-GPU setups and requires dependencies such as flash-attn, accelerate, diffusers, and transformers.
  
@@ -78,7 +87,7 @@ ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Docker
 **Text Training Data Size:** Less than a Billion Tokens  
 **Data Collection Method by dataset:** Hybrid: Automated/Synthetic  
 **Labeling Method by dataset:** Automated  
-**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Multimodal dataset combining 3D reconstruction data from DL3DV-10K (the DL3DV-ALL-960P release) with text captions. Includes sparse 3D point clouds, camera parameters, and RGB images. Camera poses are estimated with COLMAP, reconstructions are produced with 3DGUT (MCMC densification, via the 3DGRUT library), text captions are generated with a vision-language model (Qwen3-VL-30B-A3B-Instruct), and metric scale is estimated with MoGe. The dataset supports training for 3D reconstruction tasks with both real and generated data. Both variants are trained on the same dataset.
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Multimodal dataset combining 3D reconstruction data from DL3DV-10K (the DL3DV-ALL-960P release) with text captions. Includes sparse 3D point clouds, camera parameters, and RGB images. Camera poses are estimated with COLMAP, reconstructions are produced with 3DGUT (MCMC densification, via the 3DGRUT library), text captions are generated with a vision-language model (Qwen3-VL-30B-A3B-Instruct), and metric scale is estimated with MoGe. The dataset supports training for 3D reconstruction tasks with both real and generated data. The Wan2.1 14B, Wan2.1 1.3B, and Cosmos3-Nano variants are trained on this dataset.
  
 ### Testing Dataset:
  
@@ -88,7 +97,7 @@ ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Docker
  
 ### Evaluation Dataset:
  
-All numbers below are measured with the released causal auto-regressive checkpoints (`artifixer-14b.pt`, `artifixer-1.3b.pt`). Artifact removal on the Nerfbusters and DL3DV benchmarks (Difix3D+ protocol; NB = Nerfbusters):
+The tables identify the evaluated model variant and inference method. Artifact removal on the Nerfbusters and DL3DV benchmarks uses the released Wan causal auto-regressive checkpoints (`artifixer-14b.pt`, `artifixer-1.3b.pt`) with the Difix3D+ protocol (NB = Nerfbusters):
  
 | Method | NB PSNR↑ | NB SSIM↑ | NB LPIPS↓ | NB FID↓ | DL3DV PSNR↑ | DL3DV SSIM↑ | DL3DV LPIPS↓ | DL3DV FID↓ |
 | --- | --- | --- | --- | --- | --- | --- | --- | --- |
@@ -112,6 +121,23 @@ Novel-content generation on DL3DV (our evaluation split), direct ArtiFixer:
 | 1.3B | 19.12 | 0.629 | 0.332 | 16.24 |
 | 14B | 20.27 | 0.657 | 0.301 | 12.33 |
  
+Cosmos3-Nano checkpoint evaluation on DL3DV (our split), direct ArtiFixer:
+
+| Checkpoint | Sampling | PSNR↑ | SSIM↑ | LPIPS↓ | FID↓ |
+| --- | --- | --- | --- | --- | --- |
+| Cosmos3-Nano S1 (step 25,000) | Bidirectional, 50 steps, CFG 5 | 19.17 | 0.629 | 0.359 | 13.47 |
+| Cosmos3-Nano S3 (EMA700) | KV cache, 4 steps | 19.58 | 0.629 | 0.348 | 11.75 |
+
+These results use six reference views, 28 scenes, and 9,619 held-out frames.
+PSNR, SSIM, and LPIPS are averaged over frames; FID is computed over the pooled
+frames. LPIPS uses standard VGG normalization with inputs in `[-1, 1]`. The historical Wan
+measurements above retain their original evaluation protocol.
+
+Both exported checkpoints were evaluated in BF16 on H100 80 GB GPUs, with four
+GPUs processing independent samples. S1 uses chunks of up to 81 source frames;
+S3 rolls out the full trajectory with block/window/sink sizes of 7/21/7 latent
+frames. Both use seed 42 and the pinned Cosmos3-Nano base-model revision.
+
 Additional results — Tanks & Temples and multi-view consistency (supplement) — are reported in the [paper](https://research.nvidia.com/labs/sil/projects/artifixer/assets/paper.pdf).
  
 **Data Collection Method by dataset:** Hybrid: Automated/Human  
@@ -123,6 +149,22 @@ Additional results — Tanks & Temples and multi-view consistency (supplement)
 **Acceleration Engine:** FlashAttention (FA3/FA4); PyTorch SDPA (cuDNN) fallback on Ampere  
 **Test Hardware:** NVIDIA A100 80GB, NVIDIA H100, NVIDIA GB200 (Blackwell)
  
+### Cosmos3-Nano checkpoints
+
+| Checkpoint | Training stage | Sampling |
+| --- | --- | --- |
+| [`artifixer-s1-cosmos3-nano.pt`](./artifixer-s1-cosmos3-nano.pt) | S1 bidirectional teacher, step 25,000 | Bidirectional, 50 steps, text guidance 5 |
+| [`artifixer-cosmos3-nano.pt`](./artifixer-cosmos3-nano.pt) | S3 DMD, EMA step 700 | Causal KV cache, 4 steps; block/window/sink = 7/21/7 latent frames |
+
+Keep each checkpoint's `.metadata.json` sidecar beside its `.pt` file. Each
+FP32 checkpoint is approximately 63.2 GB; BF16 transformer weights alone need
+31.6 GB, before the VAE, activations, and KV cache.
+
+See the [Cosmos3-Nano inference guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/inference.md#cosmos3-nano)
+for setup, downloads, and S1/S3 commands, and the
+[training guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/training.md)
+for the three-stage recipe.
+
 ## Ethical Considerations:
  
 NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.  

SHA256SUMS

@@ -0,0 +1,7 @@
+3558f6174889ae9764746c86388343991858d44795f63196dbbe13837722782e  artifixer-cosmos3-nano.metadata.json
+3e5a648411794413d91cc78131261eea51fdba54fa8789a5bb298e460f27725a  artifixer-cosmos3-nano.pt
+4b48a7d51c5b7addac25662d08a789f7abfdb50d8d2aa2691ca65d743fe6b027  artifixer-s1-cosmos3-nano.metadata.json
+9388aeb9bc1b3b6ff2d566c255c3a8308415d706f6d55477df335e879ae8d95c  artifixer-s1-cosmos3-nano.pt
+90238f0aa41aadd4b4fd65ddb15efbefc42291cf23adc81033c5d68d7567e745  licenses/NOTICE.txt
+4ff203c3f7997c7fed287a463d733f794934a79cfabb2936008fca0bcc8ad3d6  licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf
+be21b29b1c69af5b4a4e60a6bd7bc933eec374e52be29d1ebb9e6a17a6588491  licenses/OpenMDW-1.1.txt

artifixer-cosmos3-nano.metadata.json

@@ -0,0 +1,12 @@
+{
+  "base_model_revision": "23314034b82f46b45339035dba67c3ee9bbcb8ba",
+  "camera_pose_alignment": "last",
+  "camera_ray_reference": "first",
+  "checkpoint": 700,
+  "export_date": "2026-10-21T04:47:41.238085+00:00",
+  "inference_pipeline": "kv_cache",
+  "model_id": "nvidia/Cosmos3-Nano",
+  "num_inference_steps": 4,
+  "training_stage": "dmd",
+  "weight_type": "ema"
+}

artifixer-cosmos3-nano.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e5a648411794413d91cc78131261eea51fdba54fa8789a5bb298e460f27725a
+size 63212165917

artifixer-s1-cosmos3-nano.metadata.json

@@ -0,0 +1,12 @@
+{
+  "base_model_revision": "23314034b82f46b45339035dba67c3ee9bbcb8ba",
+  "camera_pose_alignment": "last",
+  "camera_ray_reference": "first",
+  "checkpoint": 25000,
+  "export_date": "2026-10-21T04:33:05.795365+00:00",
+  "inference_pipeline": "bidirectional",
+  "model_id": "nvidia/Cosmos3-Nano",
+  "num_inference_steps": 50,
+  "training_stage": "bidirectional",
+  "weight_type": "raw"
+}

artifixer-s1-cosmos3-nano.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9388aeb9bc1b3b6ff2d566c255c3a8308415d706f6d55477df335e879ae8d95c
+size 63212170125

licenses/NOTICE.txt

@@ -0,0 +1,9 @@
+ArtiFixer Cosmos3-Nano S1 and S3 checkpoints
+
+Use of these models is governed by the NVIDIA License included in
+NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf.
+
+These models are based on NVIDIA Cosmos3-Nano, governed by the OpenMDW
+License Agreement, version 1.1, included in OpenMDW-1.1.txt.
+Base model: https://huggingface.co/nvidia/Cosmos3-Nano
+Base revision: 23314034b82f46b45339035dba67c3ee9bbcb8ba

licenses/OpenMDW-1.1.txt

@@ -0,0 +1,19 @@
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree to its terms.
+
+As used in this agreement, “Model Materials” means the materials provided to you under this agreement, consisting of: (1) one or more machine learning models (including architecture and parameters); and (2) all related artifacts (including associated data, documentation and software) that are provided to you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted, free of charge, to deal in the Model Materials without restriction, including under all copyright, patent, database, and trade secret rights included or embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your distribution (1) a copy of this agreement, and (2) all copyright notices and other notices of origin included in the Model Materials that are applicable to your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any person or entity asserting that the Model Materials directly or indirectly infringe any patent or copyright, then all rights and grants made to you hereunder are terminated, unless that lawsuit was in response to a corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to any use, modification, or sharing of any outputs generated by using the Model Materials.
+
+THE MODEL MATERIALS ARE PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION ANY PERSON’S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE USE THEREOF OR OTHER DEALINGS THEREIN.