Add stage-1 bidirectional teacher checkpoints to the model card

#5
by rdelutio - opened
Files changed (2) hide show
  1. README.md +7 -5
  2. config.json +8 -0
README.md CHANGED
@@ -8,6 +8,7 @@ library_name: artifixer
8
 
9
  ArtiFixer is a few-step causal auto-regressive model that enhances and extends 3D reconstruction. The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and Self-Forcing-style DMD distillation.
10
  ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and is released in two variants: a 14B variant based on Wan2.1's 14B model, and a lighter-weight 1.3B variant based on Wan2.1's 1.3B model that trades some reconstruction quality for significantly higher throughput and lower training and inference compute.
 
11
  _This model is for research and development only._
12
 
13
  ### License/Terms of Use:
@@ -25,7 +26,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
25
 
26
  ### Release Date:
27
 
28
- **Other:** Hugging Face: 06/04/2026 (14B variant) and 07/21/2026 (1.3B variant) via https://research.nvidia.com/labs/sil/projects/artifixer/
29
 
30
  ## Reference(s):
31
 
@@ -36,7 +37,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
36
  **Architecture Type:** Transformer
37
  **Network Architecture:** ArtifixerTransformer (built on Wan2.1's WanTransformer3DModel)
38
  **This model was developed based on Wan-AI/Wan2.1-T2V-14B-Diffusers (14B variant) and Wan-AI/Wan2.1-T2V-1.3B-Diffusers (1.3B variant).**
39
- **Number of model parameters:** 14B variant: ~16.9B trainable (16,910,955,584); 1.3B variant: ~1.68B trainable (1,678,749,760)
40
 
41
  ## Input:
42
 
@@ -50,7 +51,7 @@ Developers and researchers working on 3D reconstruction, diffusion models, and a
50
  **Output Type(s):** Image
51
  **Output Format:** RGB (Red, Green, Blue)
52
  **Output Parameters:** Two-Dimensional (2D) image frames
53
- **Other Properties Related to Output:** Generates enhanced images via a few-step causal auto-regressive diffusion model. At 4 denoising steps on a single GPU, the 14B variant reaches 8.36 FPS and the 1.3B variant 34.38 FPS; with 1 denoising step and 4-GPU context parallelism, the 1.3B variant reaches up to 101.77 FPS.
54
 
55
  Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
56
 
@@ -64,7 +65,8 @@ The integration of foundation and fine-tuned models into AI systems requires add
64
 
65
  ## Model Version(s):
66
 
67
- ArtiFixer 14B v1.0, ArtiFixer 1.3B v1.0
 
68
  ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Dockerfiles for CUDA 12 and 13, supporting both x86_64 and aarch64 architectures. The model can be run using torchrun (or accelerate) with multi-GPU setups and requires dependencies such as flash-attn, accelerate, diffusers, and transformers.
69
 
70
  ## Training, Testing, and Evaluation Datasets:
@@ -86,7 +88,7 @@ ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Docker
86
 
87
  ### Evaluation Dataset:
88
 
89
- All numbers below are measured with the released checkpoints. Artifact removal on the Nerfbusters and DL3DV benchmarks (Difix3D+ protocol; NB = Nerfbusters):
90
 
91
  | Method | NB PSNR↑ | NB SSIM↑ | NB LPIPS↓ | NB FID↓ | DL3DV PSNR↑ | DL3DV SSIM↑ | DL3DV LPIPS↓ | DL3DV FID↓ |
92
  | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 
8
 
9
  ArtiFixer is a few-step causal auto-regressive model that enhances and extends 3D reconstruction. The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and Self-Forcing-style DMD distillation.
10
  ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and is released in two variants: a 14B variant based on Wan2.1's 14B model, and a lighter-weight 1.3B variant based on Wan2.1's 1.3B model that trades some reconstruction quality for significantly higher throughput and lower training and inference compute.
11
+ In addition to the distilled few-step causal auto-regressive models, this repository also provides the bidirectional diffusion **teacher** checkpoints from which each released model was distilled (`artifixer-s1-14b.pt` and `artifixer-s1-1.3b.pt`, the stage-1 bidirectional training outputs). The teachers use full bidirectional attention with standard multi-step diffusion sampling instead of few-step causal auto-regressive rollout. They are provided for research on distillation, sampling, and long-rollout behavior, and as the starting point for reproducing the diffusion-forcing and DMD distillation stages.
12
  _This model is for research and development only._
13
 
14
  ### License/Terms of Use:
 
26
 
27
  ### Release Date:
28
 
29
+ **Other:** Hugging Face: 06/04/2026 (14B variant), 07/21/2026 (1.3B variant), and 09/2026 (bidirectional teacher checkpoints) via https://research.nvidia.com/labs/sil/projects/artifixer/
30
 
31
  ## Reference(s):
32
 
 
37
  **Architecture Type:** Transformer
38
  **Network Architecture:** ArtifixerTransformer (built on Wan2.1's WanTransformer3DModel)
39
  **This model was developed based on Wan-AI/Wan2.1-T2V-14B-Diffusers (14B variant) and Wan-AI/Wan2.1-T2V-1.3B-Diffusers (1.3B variant).**
40
+ **Number of model parameters:** 14B variant: ~16.9B trainable (16,910,955,584); 1.3B variant: ~1.68B trainable (1,678,749,760). The teacher checkpoints share the architecture and parameter counts of their respective variants.
41
 
42
  ## Input:
43
 
 
51
  **Output Type(s):** Image
52
  **Output Format:** RGB (Red, Green, Blue)
53
  **Output Parameters:** Two-Dimensional (2D) image frames
54
+ **Other Properties Related to Output:** Generates enhanced images via a few-step causal auto-regressive diffusion model. At 4 denoising steps on a single GPU, the 14B variant reaches 8.36 FPS and the 1.3B variant 34.38 FPS; with 1 denoising step and 4-GPU context parallelism, the 1.3B variant reaches up to 101.77 FPS. The bidirectional teacher checkpoints instead run standard multi-step diffusion sampling with full bidirectional attention; the FPS figures above apply to the distilled causal auto-regressive models only.
55
 
56
  Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
57
 
 
65
 
66
  ## Model Version(s):
67
 
68
+ ArtiFixer 14B v1.0, ArtiFixer 1.3B v1.0, ArtiFixer 14B Teacher v1.0, ArtiFixer 1.3B Teacher v1.0
69
+ The teacher checkpoints are the stage-1 bidirectional models from which the corresponding released causal auto-regressive models were distilled.
70
  ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Dockerfiles for CUDA 12 and 13, supporting both x86_64 and aarch64 architectures. The model can be run using torchrun (or accelerate) with multi-GPU setups and requires dependencies such as flash-attn, accelerate, diffusers, and transformers.
71
 
72
  ## Training, Testing, and Evaluation Datasets:
 
88
 
89
  ### Evaluation Dataset:
90
 
91
+ All numbers below are measured with the released causal auto-regressive checkpoints (`artifixer-14b.pt`, `artifixer-1.3b.pt`). Artifact removal on the Nerfbusters and DL3DV benchmarks (Difix3D+ protocol; NB = Nerfbusters):
92
 
93
  | Method | NB PSNR↑ | NB SSIM↑ | NB LPIPS↓ | NB FID↓ | DL3DV PSNR↑ | DL3DV SSIM↑ | DL3DV LPIPS↓ | DL3DV FID↓ |
94
  | --- | --- | --- | --- | --- | --- | --- | --- | --- |
config.json CHANGED
@@ -12,6 +12,14 @@
12
  "artifixer-1.3b": {
13
  "weight_file": "artifixer-1.3b.pt",
14
  "base_model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
 
 
 
 
 
 
 
 
15
  }
16
  },
17
  "intended_use": "3D reconstruction enhancement and extension",
 
12
  "artifixer-1.3b": {
13
  "weight_file": "artifixer-1.3b.pt",
14
  "base_model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
15
+ },
16
+ "artifixer-s1-14b": {
17
+ "weight_file": "artifixer-s1-14b.pt",
18
+ "base_model": "Wan-AI/Wan2.1-T2V-14B-Diffusers"
19
+ },
20
+ "artifixer-s1-1.3b": {
21
+ "weight_file": "artifixer-s1-1.3b.pt",
22
+ "base_model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
23
  }
24
  },
25
  "intended_use": "3D reconstruction enhancement and extension",