Add ArtiFixer 1.3B variant to the model card
#4
by rdelutio - opened
- README.md +30 -14
- config.json +11 -1
README.md
CHANGED
|
@@ -7,13 +7,13 @@ library_name: artifixer
|
|
| 7 |
## Description:
|
| 8 |
|
| 9 |
ArtiFixer is a few-step causal auto-regressive model that enhances and extends 3D reconstruction. The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and Self-Forcing-style DMD distillation.
|
| 10 |
-
ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and based on Wan2.1's 14B model.
|
| 11 |
_This model is for research and development only._
|
| 12 |
|
| 13 |
### License/Terms of Use:
|
| 14 |
|
| 15 |
GOVERNING DOWNLOAD TERMS: Use of the model is governed by the [NVIDIA License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=).
|
| 16 |
-
ADDITIONAL INFORMATION: The Wan 2.1 14B base
|
| 17 |
|
| 18 |
### Deployment Geography:
|
| 19 |
|
|
@@ -25,7 +25,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
|
|
| 25 |
|
| 26 |
### Release Date:
|
| 27 |
|
| 28 |
-
**Other:** Hugging Face: 06/04/2026 via https://research.nvidia.com/labs/sil/projects/artifixer/
|
| 29 |
|
| 30 |
## Reference(s):
|
| 31 |
|
|
@@ -35,8 +35,8 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
|
|
| 35 |
|
| 36 |
**Architecture Type:** Transformer
|
| 37 |
**Network Architecture:** ArtifixerTransformer (built on Wan2.1's WanTransformer3DModel)
|
| 38 |
-
**This model was developed based on Wan-AI/Wan2.1-T2V-14B-Diffusers.**
|
| 39 |
-
**Number of model parameters:** ~16.9B trainable (16,910,955,584)
|
| 40 |
|
| 41 |
## Input:
|
| 42 |
|
|
@@ -50,7 +50,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
|
|
| 50 |
**Output Type(s):** Image
|
| 51 |
**Output Format:** RGB (Red, Green, Blue)
|
| 52 |
**Output Parameters:** Two-Dimensional (2D) image frames
|
| 53 |
-
**Other Properties Related to Output:** Generates enhanced images via a few-step causal auto-regressive diffusion model.
|
| 54 |
|
| 55 |
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
|
| 56 |
|
|
@@ -64,7 +64,7 @@ The integration of foundation and fine-tuned models into AI systems requires add
|
|
| 64 |
|
| 65 |
## Model Version(s):
|
| 66 |
|
| 67 |
-
ArtiFixer v1.0
|
| 68 |
ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Dockerfiles for CUDA 12 and 13, supporting both x86_64 and aarch64 architectures. The model can be run using torchrun (or accelerate) with multi-GPU setups and requires dependencies such as flash-attn, accelerate, diffusers, and transformers.
|
| 69 |
|
| 70 |
## Training, Testing, and Evaluation Datasets:
|
|
@@ -76,7 +76,7 @@ ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Docker
|
|
| 76 |
**Text Training Data Size:** Less than a Billion Tokens
|
| 77 |
**Data Collection Method by dataset:** Hybrid: Automated/Synthetic
|
| 78 |
**Labeling Method by dataset:** Automated
|
| 79 |
-
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Multimodal dataset combining 3D reconstruction data from DL3DV-10K (the DL3DV-ALL-960P release) with text captions. Includes sparse 3D point clouds, camera parameters, and RGB images. Camera poses are estimated with COLMAP, reconstructions are produced with 3DGUT (MCMC densification, via the 3DGRUT library), text captions are generated with a vision-language model (Qwen3-VL-30B-A3B-Instruct), and metric scale is estimated with MoGe. The dataset supports training for 3D reconstruction tasks with both real and generated data.
|
| 80 |
|
| 81 |
### Testing Dataset:
|
| 82 |
|
|
@@ -86,19 +86,35 @@ ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Docker
|
|
| 86 |
|
| 87 |
### Evaluation Dataset:
|
| 88 |
|
| 89 |
-
Artifact removal on the Nerfbusters and DL3DV benchmarks (Difix3D+ protocol; NB = Nerfbusters):
|
| 90 |
|
| 91 |
| Method | NB PSNR↑ | NB SSIM↑ | NB LPIPS↓ | NB FID↓ | DL3DV PSNR↑ | DL3DV SSIM↑ | DL3DV LPIPS↓ | DL3DV FID↓ |
|
| 92 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 93 |
-
| ArtiFixer |
|
| 94 |
-
| ArtiFixer 3D | 20.
|
| 95 |
-
| ArtiFixer 3D+ | 20.
|
|
|
|
| 96 |
|
| 97 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
**Data Collection Method by dataset:** Hybrid: Automated/Human
|
| 100 |
**Labeling Method by dataset:** Hybrid: Automated/Human
|
| 101 |
-
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Evaluated on diverse 3D reconstruction benchmarks including DL3DV and
|
| 102 |
|
| 103 |
## Inference:
|
| 104 |
|
|
|
|
| 7 |
## Description:
|
| 8 |
|
| 9 |
ArtiFixer is a few-step causal auto-regressive model that enhances and extends 3D reconstruction. The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and Self-Forcing-style DMD distillation.
|
| 10 |
+
ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and is released in two variants: a 14B variant based on Wan2.1's 14B model, and a lighter-weight 1.3B variant based on Wan2.1's 1.3B model that trades some reconstruction quality for significantly higher throughput and lower training and inference compute.
|
| 11 |
_This model is for research and development only._
|
| 12 |
|
| 13 |
### License/Terms of Use:
|
| 14 |
|
| 15 |
GOVERNING DOWNLOAD TERMS: Use of the model is governed by the [NVIDIA License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=).
|
| 16 |
+
ADDITIONAL INFORMATION: The Wan 2.1 14B and 1.3B base models are governed by the [Apache License, Version 2.0.](https://www.apache.org/licenses/LICENSE-2.0)
|
| 17 |
|
| 18 |
### Deployment Geography:
|
| 19 |
|
|
|
|
| 25 |
|
| 26 |
### Release Date:
|
| 27 |
|
| 28 |
+
**Other:** Hugging Face: 06/04/2026 (14B variant) and 07/21/2026 (1.3B variant) via https://research.nvidia.com/labs/sil/projects/artifixer/
|
| 29 |
|
| 30 |
## Reference(s):
|
| 31 |
|
|
|
|
| 35 |
|
| 36 |
**Architecture Type:** Transformer
|
| 37 |
**Network Architecture:** ArtifixerTransformer (built on Wan2.1's WanTransformer3DModel)
|
| 38 |
+
**This model was developed based on Wan-AI/Wan2.1-T2V-14B-Diffusers (14B variant) and Wan-AI/Wan2.1-T2V-1.3B-Diffusers (1.3B variant).**
|
| 39 |
+
**Number of model parameters:** 14B variant: ~16.9B trainable (16,910,955,584); 1.3B variant: ~1.68B trainable (1,678,749,760)
|
| 40 |
|
| 41 |
## Input:
|
| 42 |
|
|
|
|
| 50 |
**Output Type(s):** Image
|
| 51 |
**Output Format:** RGB (Red, Green, Blue)
|
| 52 |
**Output Parameters:** Two-Dimensional (2D) image frames
|
| 53 |
+
**Other Properties Related to Output:** Generates enhanced images via a few-step causal auto-regressive diffusion model. At 4 denoising steps on a single GPU, the 14B variant reaches 8.36 FPS and the 1.3B variant 34.38 FPS; with 1 denoising step and 4-GPU context parallelism, the 1.3B variant reaches up to 101.77 FPS.
|
| 54 |
|
| 55 |
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
|
| 56 |
|
|
|
|
| 64 |
|
| 65 |
## Model Version(s):
|
| 66 |
|
| 67 |
+
ArtiFixer 14B v1.0, ArtiFixer 1.3B v1.0
|
| 68 |
ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Dockerfiles for CUDA 12 and 13, supporting both x86_64 and aarch64 architectures. The model can be run using torchrun (or accelerate) with multi-GPU setups and requires dependencies such as flash-attn, accelerate, diffusers, and transformers.
|
| 69 |
|
| 70 |
## Training, Testing, and Evaluation Datasets:
|
|
|
|
| 76 |
**Text Training Data Size:** Less than a Billion Tokens
|
| 77 |
**Data Collection Method by dataset:** Hybrid: Automated/Synthetic
|
| 78 |
**Labeling Method by dataset:** Automated
|
| 79 |
+
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Multimodal dataset combining 3D reconstruction data from DL3DV-10K (the DL3DV-ALL-960P release) with text captions. Includes sparse 3D point clouds, camera parameters, and RGB images. Camera poses are estimated with COLMAP, reconstructions are produced with 3DGUT (MCMC densification, via the 3DGRUT library), text captions are generated with a vision-language model (Qwen3-VL-30B-A3B-Instruct), and metric scale is estimated with MoGe. The dataset supports training for 3D reconstruction tasks with both real and generated data. Both variants are trained on the same dataset.
|
| 80 |
|
| 81 |
### Testing Dataset:
|
| 82 |
|
|
|
|
| 86 |
|
| 87 |
### Evaluation Dataset:
|
| 88 |
|
| 89 |
+
All numbers below are measured with the released checkpoints. Artifact removal on the Nerfbusters and DL3DV benchmarks (Difix3D+ protocol; NB = Nerfbusters):
|
| 90 |
|
| 91 |
| Method | NB PSNR↑ | NB SSIM↑ | NB LPIPS↓ | NB FID↓ | DL3DV PSNR↑ | DL3DV SSIM↑ | DL3DV LPIPS↓ | DL3DV FID↓ |
|
| 92 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 93 |
+
| ArtiFixer (14B) | 20.66 | 0.725 | 0.245 | 30.30 | 20.25 | 0.688 | 0.234 | 18.93 |
|
| 94 |
+
| ArtiFixer 3D (14B) | 20.76 | 0.745 | 0.260 | 34.70 | 20.43 | 0.719 | 0.252 | 21.72 |
|
| 95 |
+
| ArtiFixer 3D+ (14B) | 20.76 | 0.734 | 0.255 | 33.58 | 20.39 | 0.700 | 0.247 | 21.17 |
|
| 96 |
+
| ArtiFixer (1.3B) | 19.95 | 0.716 | 0.267 | 40.09 | 19.34 | 0.667 | 0.265 | 25.32 |
|
| 97 |
|
| 98 |
+
Sparse-view reconstruction on Mip-NeRF 360 (Cat3D protocol), direct ArtiFixer:
|
| 99 |
+
|
| 100 |
+
| Views | 1.3B PSNR↑ | 1.3B SSIM↑ | 1.3B LPIPS↓ | 14B PSNR↑ | 14B SSIM↑ | 14B LPIPS↓ |
|
| 101 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 102 |
+
| 3-view | 16.73 | 0.420 | 0.466 | 17.58 | 0.442 | 0.436 |
|
| 103 |
+
| 6-view | 18.18 | 0.469 | 0.413 | 19.12 | 0.495 | 0.385 |
|
| 104 |
+
| 9-view | 19.47 | 0.511 | 0.371 | 20.39 | 0.534 | 0.351 |
|
| 105 |
+
|
| 106 |
+
Novel-content generation on DL3DV (our evaluation split), direct ArtiFixer:
|
| 107 |
+
|
| 108 |
+
| Variant | PSNR↑ | SSIM↑ | LPIPS↓ | FID↓ |
|
| 109 |
+
| --- | --- | --- | --- | --- |
|
| 110 |
+
| 1.3B | 19.12 | 0.629 | 0.332 | 16.24 |
|
| 111 |
+
| 14B | 20.27 | 0.657 | 0.301 | 12.33 |
|
| 112 |
+
|
| 113 |
+
Additional results — Tanks & Temples and multi-view consistency (supplement) — are reported in the [paper](https://research.nvidia.com/labs/sil/projects/artifixer/assets/paper.pdf).
|
| 114 |
|
| 115 |
**Data Collection Method by dataset:** Hybrid: Automated/Human
|
| 116 |
**Labeling Method by dataset:** Hybrid: Automated/Human
|
| 117 |
+
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Evaluated on diverse 3D reconstruction benchmarks including DL3DV, Nerfbusters, and Mip-NeRF 360, assessing performance on held-out validation frames, full source trajectories, and prepared trajectories. Metrics include reconstruction quality, multi-view consistency, and inference speed.
|
| 118 |
|
| 119 |
## Inference:
|
| 120 |
|
config.json
CHANGED
|
@@ -4,6 +4,16 @@
|
|
| 4 |
"framework": "pytorch",
|
| 5 |
"weight_file": "artifixer-14b.pt",
|
| 6 |
"base_model": "Wan-AI/Wan2.1-T2V-14B-Diffusers",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
"intended_use": "3D reconstruction enhancement and extension",
|
| 8 |
-
"download_counting_note": "This metadata file is included so Hugging Face Hub download statistics can count repository downloads through the default config.json query file. Model weights are stored in
|
| 9 |
}
|
|
|
|
| 4 |
"framework": "pytorch",
|
| 5 |
"weight_file": "artifixer-14b.pt",
|
| 6 |
"base_model": "Wan-AI/Wan2.1-T2V-14B-Diffusers",
|
| 7 |
+
"variants": {
|
| 8 |
+
"artifixer-14b": {
|
| 9 |
+
"weight_file": "artifixer-14b.pt",
|
| 10 |
+
"base_model": "Wan-AI/Wan2.1-T2V-14B-Diffusers"
|
| 11 |
+
},
|
| 12 |
+
"artifixer-1.3b": {
|
| 13 |
+
"weight_file": "artifixer-1.3b.pt",
|
| 14 |
+
"base_model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
"intended_use": "3D reconstruction enhancement and extension",
|
| 18 |
+
"download_counting_note": "This metadata file is included so Hugging Face Hub download statistics can count repository downloads through the default config.json query file. Model weights are stored in the .pt files listed under variants."
|
| 19 |
}
|