Keep Cosmos checkpoint usage concise in the model card
Browse filesRetain the S1/S3 checkpoint table and metadata/memory notes; link to the full inference and training guides. Evaluation results, licenses and all other model-card sections are unchanged.
README.md
CHANGED
|
@@ -149,85 +149,21 @@ Additional results — Tanks & Temples and multi-view consistency (supplement)
|
|
| 149 |
**Acceleration Engine:** FlashAttention (FA3/FA4); PyTorch SDPA (cuDNN) fallback on Ampere
|
| 150 |
**Test Hardware:** NVIDIA A100 80GB, NVIDIA H100, NVIDIA GB200 (Blackwell)
|
| 151 |
|
| 152 |
-
## Cosmos3-Nano checkpoints
|
| 153 |
|
| 154 |
| Checkpoint | Training stage | Sampling |
|
| 155 |
| --- | --- | --- |
|
| 156 |
| [`artifixer-s1-cosmos3-nano.pt`](./artifixer-s1-cosmos3-nano.pt) | S1 bidirectional teacher, step 25,000 | Bidirectional, 50 steps, text guidance 5 |
|
| 157 |
| [`artifixer-cosmos3-nano.pt`](./artifixer-cosmos3-nano.pt) | S3 DMD, EMA step 700 | Causal KV cache, 4 steps; block/window/sink = 7/21/7 latent frames |
|
| 158 |
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
The [training guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/training.md)
|
| 163 |
-
describes the three-stage recipe and the checkpoint handoffs.
|
| 164 |
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
host FFmpeg supplied as described in the
|
| 170 |
-
[setup guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/setup.md).
|
| 171 |
-
|
| 172 |
-
Download both checkpoints and their required metadata sidecars:
|
| 173 |
-
|
| 174 |
-
```bash
|
| 175 |
-
export CHECKPOINT_ROOT=/data/artifixer-checkpoints
|
| 176 |
-
hf download nvidia/ArtiFixer \
|
| 177 |
-
artifixer-cosmos3-nano.pt artifixer-cosmos3-nano.metadata.json \
|
| 178 |
-
artifixer-s1-cosmos3-nano.pt artifixer-s1-cosmos3-nano.metadata.json \
|
| 179 |
-
licenses/NOTICE.txt licenses/OpenMDW-1.1.txt \
|
| 180 |
-
licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf \
|
| 181 |
-
SHA256SUMS --local-dir "$CHECKPOINT_ROOT"
|
| 182 |
-
(cd "$CHECKPOINT_ROOT" && sha256sum --check SHA256SUMS)
|
| 183 |
-
```
|
| 184 |
-
|
| 185 |
-
Prepare a calibrated COLMAP scene using the repository's input-layout guide.
|
| 186 |
-
Use the actual source FPS (60 in this DL3DV example), a fresh output root,
|
| 187 |
-
and Cosmos token-ID prompts:
|
| 188 |
-
|
| 189 |
-
```bash
|
| 190 |
-
export COLMAP_SCENE=/data/colmap-scenes/my_scene
|
| 191 |
-
export SCENE_ROOT=/data/artifixer-prep/my_scene-cosmos3
|
| 192 |
-
python -m data_processing.prepare_colmap_artifixer_inputs \
|
| 193 |
-
--colmap_dir "$COLMAP_SCENE" --output_root "$SCENE_ROOT" \
|
| 194 |
-
--captioning_model_id Qwen/Qwen3-VL-4B-Instruct \
|
| 195 |
-
--prompt_format cosmos3_omni_token_ids --dataset_fps 60
|
| 196 |
-
|
| 197 |
-
python -m model_eval.run_inference \
|
| 198 |
-
--evalset reconstructed_colmap \
|
| 199 |
-
--checkpoint "$CHECKPOINT_ROOT/artifixer-cosmos3-nano.pt" \
|
| 200 |
-
--model_id nvidia/Cosmos3-Nano \
|
| 201 |
-
--model_revision 23314034b82f46b45339035dba67c3ee9bbcb8ba \
|
| 202 |
-
--split_path "$SCENE_ROOT/split.json" --render_trajectory all_frames \
|
| 203 |
-
--save_dir /data/artifixer-corrected --render_name cosmos3-nano-s3 \
|
| 204 |
-
--inference_pipeline kv_cache --num_inference_steps 4 \
|
| 205 |
-
--frames_per_block 7 --local_attn_size 21 --sink_size 7 \
|
| 206 |
-
--camera_pose_alignment last --seed 42
|
| 207 |
-
```
|
| 208 |
-
|
| 209 |
-
For the S1 teacher, use the same prepared inputs with bidirectional sampling:
|
| 210 |
-
|
| 211 |
-
```bash
|
| 212 |
-
python -m model_eval.run_inference \
|
| 213 |
-
--evalset reconstructed_colmap \
|
| 214 |
-
--checkpoint "$CHECKPOINT_ROOT/artifixer-s1-cosmos3-nano.pt" \
|
| 215 |
-
--model_id nvidia/Cosmos3-Nano \
|
| 216 |
-
--model_revision 23314034b82f46b45339035dba67c3ee9bbcb8ba \
|
| 217 |
-
--split_path "$SCENE_ROOT/split.json" --render_trajectory all_frames \
|
| 218 |
-
--save_dir /data/artifixer-corrected --render_name cosmos3-nano-s1 \
|
| 219 |
-
--inference_pipeline bidirectional --num_inference_steps 50 \
|
| 220 |
-
--text_guidance_scale 5 --bidirectional_chunk_size 81 \
|
| 221 |
-
--camera_pose_alignment last --seed 42
|
| 222 |
-
```
|
| 223 |
-
|
| 224 |
-
Keep each metadata sidecar beside its weights. The camera-pose alignment is
|
| 225 |
-
`last`; camera rays use the first target camera as their fixed origin. The model
|
| 226 |
-
profile selects flow shift 5 and RoPE FPS 24; source FPS controls prompt timing.
|
| 227 |
-
Wan T5 embedding files are incompatible with Cosmos token-ID prompts. Use a new
|
| 228 |
-
render name for each setting to avoid reusing completed outputs. Text guidance
|
| 229 |
-
is supported by the S1 bidirectional pipeline; S3 KV-cache sampling does not use
|
| 230 |
-
text CFG.
|
| 231 |
|
| 232 |
## Ethical Considerations:
|
| 233 |
|
|
|
|
| 149 |
**Acceleration Engine:** FlashAttention (FA3/FA4); PyTorch SDPA (cuDNN) fallback on Ampere
|
| 150 |
**Test Hardware:** NVIDIA A100 80GB, NVIDIA H100, NVIDIA GB200 (Blackwell)
|
| 151 |
|
| 152 |
+
### Cosmos3-Nano checkpoints
|
| 153 |
|
| 154 |
| Checkpoint | Training stage | Sampling |
|
| 155 |
| --- | --- | --- |
|
| 156 |
| [`artifixer-s1-cosmos3-nano.pt`](./artifixer-s1-cosmos3-nano.pt) | S1 bidirectional teacher, step 25,000 | Bidirectional, 50 steps, text guidance 5 |
|
| 157 |
| [`artifixer-cosmos3-nano.pt`](./artifixer-cosmos3-nano.pt) | S3 DMD, EMA step 700 | Causal KV cache, 4 steps; block/window/sink = 7/21/7 latent frames |
|
| 158 |
|
| 159 |
+
Keep each checkpoint's `.metadata.json` sidecar beside its `.pt` file. Each
|
| 160 |
+
FP32 checkpoint is approximately 63.2 GB; BF16 transformer weights alone need
|
| 161 |
+
31.6 GB, before the VAE, activations, and KV cache.
|
|
|
|
|
|
|
| 162 |
|
| 163 |
+
See the [Cosmos3-Nano inference guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/inference.md#cosmos3-nano)
|
| 164 |
+
for setup, downloads, and S1/S3 commands, and the
|
| 165 |
+
[training guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/training.md)
|
| 166 |
+
for the three-stage recipe.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
|
| 168 |
## Ethical Considerations:
|
| 169 |
|