Add files using upload-large-folder tool

README.md

@@ -1,6 +1,115 @@
----
-license: other
-license_name: nvidia-license
-license_link: >-
-  https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=
----
+# ArtiFixer Overview
+
+## Description:  
+ArtiFixer is a few-step causal auto-regressive model that enhances and extends 3D reconstruction. The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and self forcing DMD distillation.  
+ArtiFixer was developed by NVIDIA (Spatial Intelligence Lab) and based on a Wan2.1's 14B model.  
+_This model is for research and development only._  
+
+
+### License/Terms of Use:
+GOVERNING DOWNLOAD TERMS: Use of the model is governed by the [NVIDIA License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=).
+ADDITIONAL INFORMATION: The Wan 2.1 14B base model is governed by the [Apache License, Version 2.0.](https://www.apache.org/licenses/LICENSE-2.0)
+### Deployment Geography:  
+Global
+
+### Use Case:  
+Developers and researchers working on 3D reconstruction, diffusion models, and auto-regressive techniques for enhancing and extending 3D reconstruction capabilities.  
+  
+
+### Release Date:
+**Other:** Hugging Face: 06/01/2026 via https://research.nvidia.com/labs/sil/projects/artifixer/
+  
+
+## Reference(s):
+[ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models](https://research.nvidia.com/labs/sil/projects/artifixer/assets/paper.pdf)  
+
+## Model Architecture:   
+**Architecture Type:** Transformer   
+**Network Architecture:** ArtifixerTransformerBlock  
+**This model was developed based on Wan-AI/Wan2.1-T2V-14B-Diffusers.**  
+**Number of model parameters:** 14B (1.4*10^10)  
+
+  
+ 
+## Input:  
+**Input Type(s):** Image, text
+**Input Format(s):** RGB (Red, Green, Blue), opacity maps, camera ray maps and text
+**Input Parameters:** Rendered RGB and opacity maps from underlying 3D representationm, camera ray maps and text prompt
+**Other Properties Related to Input:** Model refines and extends renderings from imperfect 3D reconstruction, requiring camera intrinsics and camera poses.  
+  
+
+## Output:  
+**Output Type(s):** Image  
+**Output Format:** RGB (Red, Green, Blue)  
+**Output Parameters:** Three-Dimensional (3D)  
+**Other Properties Related to Output:** Generates enhanced images via few-step causal auto-regressive diffusion model. 
+   
+
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
+
+## Software Integration:
+**Runtime Engine(s):**
+* TensorRT-LLM
+* vLLM
+* Transformers
+
+**Supported Hardware Microarchitecture Compatibility:**
+* NVIDIA Blackwell
+* NVIDIA Hopper
+
+**Supported Operating System(s):** Linux  
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.  
+
+
+## Model Version(s): 
+ArtiFixer v1.0  
+
+ArtiFixer integrates with PyTorch and requires CUDA environments. It uses Dockerfiles for CUDA 12 and 13, supporting both x86_64 and aarch64 architectures. The model can be run using torchrun with multi-GPU setups and requires specific dependencies like flash-attn, accelerate, diffusers, and transformers.  
+ 
+## Training, Testing, and Evaluation Datasets:  
+
+
+## Training Dataset:
+
+**Data Modality:**
+* Text  
+* Image  
+
+
+**Image Training Data Size:** Less than a Million Images  
+**Text Training Data Size:** Less than a Billion Tokens  
+**Data Collection Method by dataset:** Hybrid: Automated/Synthetic  
+**Labeling Method by dataset:** Human  
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Multimodal dataset combining 3D reconstruction data from DL3DV10K with text captions. Includes sparse 3D point clouds, camera parameters, and RGB images. Data is collected from various scenes and processed using COLMAP for reconstruction. The dataset supports training for 3D reconstruction tasks with both real and generated data.
+
+### Testing Dataset:
+
+**Data Collection Method by dataset:** Automated  
+**Labeling Method by dataset:** Automated  
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** We evaluate our model on 4 benchmarks (DL3DV, Nerfbusters, M360, Tandt). We follow standard procedures for sparse reconstruction evaluation: only use a subset of frames ie. 3, 6 or 9 to evaluate on the remainaing held out frames. We follow the protocol proposed by [Cat3D](https://cat3d.github.io/) for M360 evaluation, [Difix3D](https://research.nvidia.com/labs/toronto-ai/difix3d/) for Nerfbusters and DL3DV evaluation, [ReconX](https://liuff19.github.io/ReconX/) for TandT and finally propose our own evaluation set for DL3DV.
+
+### Evaluation Dataset:
+| Benchmark | Metric | Score |
+| --- | --- | --- |
+| M360 3view reconstruction | PSNR | 17.51 |
+| M360 6view reconstruction | PSNR | 18.95 |   
+additional evaluations in https://research.nvidia.com/labs/sil/projects/artifixer/.
+
+**Data Collection Method by dataset:** Hybrid: Automated/Human  
+**Labeling Method by dataset:** Hybrid: Automated/Human  
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Evaluated on diverse 3D reconstruction benchmarks including DL3DV and Nerfbusters, assessing performance on held-out validation frames, full source trajectories, and prepared trajectories. Metrics include reconstruction quality, multi-view consistency and inference speed.
+
+
+
+## Inference:
+**Acceleration Engine:** TensorRT-LLM, vLLM
+**Test Hardware:**
+* NVIDIA A100 80GB
+* NVIDIA H100
+
+
+## Ethical Considerations:
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+
+Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).

eval-assets/dl3dv_difix_frame_remap.json

@@ -0,0 +1,42 @@
+{
+  "description": "DL3DV DiFix eval uses the compact frame space from DL3DV-Benchmark/<scene>/nerfstudio. These frames are absent from that compact frame list but present in DL3DV-ALL-960P, so release eval drops them before applying DiFix train ids and visibility masks.",
+  "scenes": {
+    "15ff83e2531668d27c92091c97d31401ce323e24ee7c844cb32d5109ab9335f7": {
+      "dropped_file_basenames": [
+        "frame_00285.png",
+        "frame_00286.png",
+        "frame_00287.png",
+        "frame_00288.png",
+        "frame_00289.png"
+      ]
+    },
+    "280abf7bd93b81b077af1db638229dbb09869052fa0b7b57c81c94d2db893829": {
+      "dropped_file_basenames": [
+        "frame_00026.png",
+        "frame_00151.png",
+        "frame_00162.png",
+        "frame_00337.png"
+      ]
+    },
+    "5c8dafad7d782c76ffad8c14e9e1244ce2b83aa12324c54a3cc10176964acf04": {
+      "dropped_file_basenames": [
+        "frame_00077.png",
+        "frame_00078.png",
+        "frame_00079.png",
+        "frame_00080.png",
+        "frame_00084.png",
+        "frame_00085.png"
+      ]
+    },
+    "918c8dad730c3b804306c5da8486124be4aa0612e85fb825338fd350c912e1b0": {
+      "dropped_file_basenames": [
+        "frame_00072.png",
+        "frame_00073.png",
+        "frame_00074.png",
+        "frame_00075.png",
+        "frame_00076.png"
+      ]
+    }
+  },
+  "version": 1
+}

eval-assets/dl3dv_difix_test_scenes.txt

@@ -0,0 +1,27 @@
+2385549d398bdfb55ed17b547c9846f0d01d571b4e050f90df707b577084fb55
+2f3e1c0f688c84cec67f9a1ea219c54c14ffabf31a046e620dacc690cac2f1bd
+ceb252f5d4419510655cf9ed7afbf3e8e688825f798d80414c7715dc8ace153a
+d3812aad538261e7f73c75762ff55f23b468bcc76f376d52ac86ca6cf3c44b4b
+35872363e17af5d173b6a0b09fcf5de94627ad5dc5f8a9ad4c579f3e70b4797a
+50c46cf8b8b22c8d2ffdef8964b05ddbceaef312c9a9ff331d1ecebfd223f72a
+adf35184a12d4cfa3f4248b87aa5adb4f39f179df460d6d76136e13d37299a2a
+032dee9fb0a8bc1b90871dc5fe950080d0bcd3caf166447f44e60ca50ac04ec7
+eb4cf52988f805e6fce11d1b239fa9de32eb157364cff06ebac0aa50e0a46567
+493816813d2d6d248eb3c2b0b77b63e54235266e9a06e270fd0d282f13960493
+7a9f97660be8f2421b37dd413c898360532f17c6447c11515e8aded2d80eb99d
+fb3b73f1d3fe9d192f21f55f5100fd258887aef345f778e0a64fc0587930a6f9
+15ff83e2531668d27c92091c97d31401ce323e24ee7c844cb32d5109ab9335f7
+3b7529dcccf7097cba31a989536035595e214b0f3775867fd3a62eb186870090
+918c8dad730c3b804306c5da8486124be4aa0612e85fb825338fd350c912e1b0
+d4fbeba0168af8fddb2fc695881787aedcd62f477c7dcec9ebca7b8594bbd95b
+c076929db6501cf7ebe386c70e6d77ea3af844a745e794f2ec17c981c465a69b
+8b9fb9d9f10e8c64d5034be69809465753b8ba88ef12da82afd47d38ee789934
+280abf7bd93b81b077af1db638229dbb09869052fa0b7b57c81c94d2db893829
+8fdc5130f0731360cc053f59049bfe6db509567457c5471c970292966cb34f17
+5c8dafad7d782c76ffad8c14e9e1244ce2b83aa12324c54a3cc10176964acf04
+35317e621976e87f0c143e66fc61fb8cddb4ff134304da7a00e32ac1983105b4
+a401469cb0fa576b93e836a4a6d712394f1a55a0a53a42250286980bd078e669
+a62c330f5403e2e41a82a74c4e865b705c5706843b992fae2fe2e538b122d984
+946f49be73928469000baa5ca04d2573137c5ee6a66362bcf8d130354dca8924
+e8ce51b6abfe05bf8dca47e29c8be6c1e6de27a8c9fece7a121400b931b2ca0f
+70eac6ff18a1daae4eeccc5eb18723eaf7e029a77e79c79d9573f7bac59ba92b

eval-assets/dl3dv_ours_test_scenes.txt

@@ -0,0 +1,28 @@
+2385549d398bdfb55ed17b547c9846f0d01d571b4e050f90df707b577084fb55
+cd9c981eeb4a9091547af19181b382698e9d9eee0a838c7c9783a8a268af6aee
+54bf355ca7e08ed1bc86f5772e564ac0f92981ca25dab24d86b694e915fc4c43
+2f3e1c0f688c84cec67f9a1ea219c54c14ffabf31a046e620dacc690cac2f1bd
+ceb252f5d4419510655cf9ed7afbf3e8e688825f798d80414c7715dc8ace153a
+d3812aad538261e7f73c75762ff55f23b468bcc76f376d52ac86ca6cf3c44b4b
+35872363e17af5d173b6a0b09fcf5de94627ad5dc5f8a9ad4c579f3e70b4797a
+50c46cf8b8b22c8d2ffdef8964b05ddbceaef312c9a9ff331d1ecebfd223f72a
+adf35184a12d4cfa3f4248b87aa5adb4f39f179df460d6d76136e13d37299a2a
+032dee9fb0a8bc1b90871dc5fe950080d0bcd3caf166447f44e60ca50ac04ec7
+eb4cf52988f805e6fce11d1b239fa9de32eb157364cff06ebac0aa50e0a46567
+493816813d2d6d248eb3c2b0b77b63e54235266e9a06e270fd0d282f13960493
+7a9f97660be8f2421b37dd413c898360532f17c6447c11515e8aded2d80eb99d
+15ff83e2531668d27c92091c97d31401ce323e24ee7c844cb32d5109ab9335f7
+3b7529dcccf7097cba31a989536035595e214b0f3775867fd3a62eb186870090
+918c8dad730c3b804306c5da8486124be4aa0612e85fb825338fd350c912e1b0
+d4fbeba0168af8fddb2fc695881787aedcd62f477c7dcec9ebca7b8594bbd95b
+c076929db6501cf7ebe386c70e6d77ea3af844a745e794f2ec17c981c465a69b
+8b9fb9d9f10e8c64d5034be69809465753b8ba88ef12da82afd47d38ee789934
+280abf7bd93b81b077af1db638229dbb09869052fa0b7b57c81c94d2db893829
+8fdc5130f0731360cc053f59049bfe6db509567457c5471c970292966cb34f17
+5c8dafad7d782c76ffad8c14e9e1244ce2b83aa12324c54a3cc10176964acf04
+35317e621976e87f0c143e66fc61fb8cddb4ff134304da7a00e32ac1983105b4
+a401469cb0fa576b93e836a4a6d712394f1a55a0a53a42250286980bd078e669
+a62c330f5403e2e41a82a74c4e865b705c5706843b992fae2fe2e538b122d984
+946f49be73928469000baa5ca04d2573137c5ee6a66362bcf8d130354dca8924
+e8ce51b6abfe05bf8dca47e29c8be6c1e6de27a8c9fece7a121400b931b2ca0f
+70eac6ff18a1daae4eeccc5eb18723eaf7e029a77e79c79d9573f7bac59ba92b