rdelutio commited on
Commit
d1dacba
·
verified ·
1 Parent(s): 2e02524

Keep Cosmos checkpoint usage concise in the model card

Browse files

Retain the S1/S3 checkpoint table and metadata/memory notes; link to the full inference and training guides. Evaluation results, licenses and all other model-card sections are unchanged.

Files changed (1) hide show
  1. README.md +8 -72
README.md CHANGED
@@ -149,85 +149,21 @@ Additional results — Tanks & Temples and multi-view consistency (supplement)
149
  **Acceleration Engine:** FlashAttention (FA3/FA4); PyTorch SDPA (cuDNN) fallback on Ampere
150
  **Test Hardware:** NVIDIA A100 80GB, NVIDIA H100, NVIDIA GB200 (Blackwell)
151
 
152
- ## Cosmos3-Nano checkpoints and usage
153
 
154
  | Checkpoint | Training stage | Sampling |
155
  | --- | --- | --- |
156
  | [`artifixer-s1-cosmos3-nano.pt`](./artifixer-s1-cosmos3-nano.pt) | S1 bidirectional teacher, step 25,000 | Bidirectional, 50 steps, text guidance 5 |
157
  | [`artifixer-cosmos3-nano.pt`](./artifixer-cosmos3-nano.pt) | S3 DMD, EMA step 700 | Causal KV cache, 4 steps; block/window/sink = 7/21/7 latent frames |
158
 
159
- S1 is the teacher used to initialize the diffusion-forcing stage preceding S3
160
- DMD distillation. Both variants are trained on DL3DV-10K (DL3DV-ALL-960P),
161
- using sparse-view reconstructions, calibrated cameras, and text captions.
162
- The [training guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/training.md)
163
- describes the three-stage recipe and the checkpoint handoffs.
164
 
165
- Each FP32 transformer checkpoint is approximately 63.2 GB (58.9 GiB). BF16
166
- runtime transformer weights alone occupy 31.6 GB (29.4 GiB), before the VAE,
167
- activations, and KV cache. Use the matching ArtiFixer source and Docker
168
- environment from the [repository](https://github.com/nv-tlabs/ArtiFixer), with
169
- host FFmpeg supplied as described in the
170
- [setup guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/setup.md).
171
-
172
- Download both checkpoints and their required metadata sidecars:
173
-
174
- ```bash
175
- export CHECKPOINT_ROOT=/data/artifixer-checkpoints
176
- hf download nvidia/ArtiFixer \
177
- artifixer-cosmos3-nano.pt artifixer-cosmos3-nano.metadata.json \
178
- artifixer-s1-cosmos3-nano.pt artifixer-s1-cosmos3-nano.metadata.json \
179
- licenses/NOTICE.txt licenses/OpenMDW-1.1.txt \
180
- licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf \
181
- SHA256SUMS --local-dir "$CHECKPOINT_ROOT"
182
- (cd "$CHECKPOINT_ROOT" && sha256sum --check SHA256SUMS)
183
- ```
184
-
185
- Prepare a calibrated COLMAP scene using the repository's input-layout guide.
186
- Use the actual source FPS (60 in this DL3DV example), a fresh output root,
187
- and Cosmos token-ID prompts:
188
-
189
- ```bash
190
- export COLMAP_SCENE=/data/colmap-scenes/my_scene
191
- export SCENE_ROOT=/data/artifixer-prep/my_scene-cosmos3
192
- python -m data_processing.prepare_colmap_artifixer_inputs \
193
- --colmap_dir "$COLMAP_SCENE" --output_root "$SCENE_ROOT" \
194
- --captioning_model_id Qwen/Qwen3-VL-4B-Instruct \
195
- --prompt_format cosmos3_omni_token_ids --dataset_fps 60
196
-
197
- python -m model_eval.run_inference \
198
- --evalset reconstructed_colmap \
199
- --checkpoint "$CHECKPOINT_ROOT/artifixer-cosmos3-nano.pt" \
200
- --model_id nvidia/Cosmos3-Nano \
201
- --model_revision 23314034b82f46b45339035dba67c3ee9bbcb8ba \
202
- --split_path "$SCENE_ROOT/split.json" --render_trajectory all_frames \
203
- --save_dir /data/artifixer-corrected --render_name cosmos3-nano-s3 \
204
- --inference_pipeline kv_cache --num_inference_steps 4 \
205
- --frames_per_block 7 --local_attn_size 21 --sink_size 7 \
206
- --camera_pose_alignment last --seed 42
207
- ```
208
-
209
- For the S1 teacher, use the same prepared inputs with bidirectional sampling:
210
-
211
- ```bash
212
- python -m model_eval.run_inference \
213
- --evalset reconstructed_colmap \
214
- --checkpoint "$CHECKPOINT_ROOT/artifixer-s1-cosmos3-nano.pt" \
215
- --model_id nvidia/Cosmos3-Nano \
216
- --model_revision 23314034b82f46b45339035dba67c3ee9bbcb8ba \
217
- --split_path "$SCENE_ROOT/split.json" --render_trajectory all_frames \
218
- --save_dir /data/artifixer-corrected --render_name cosmos3-nano-s1 \
219
- --inference_pipeline bidirectional --num_inference_steps 50 \
220
- --text_guidance_scale 5 --bidirectional_chunk_size 81 \
221
- --camera_pose_alignment last --seed 42
222
- ```
223
-
224
- Keep each metadata sidecar beside its weights. The camera-pose alignment is
225
- `last`; camera rays use the first target camera as their fixed origin. The model
226
- profile selects flow shift 5 and RoPE FPS 24; source FPS controls prompt timing.
227
- Wan T5 embedding files are incompatible with Cosmos token-ID prompts. Use a new
228
- render name for each setting to avoid reusing completed outputs. Text guidance
229
- is supported by the S1 bidirectional pipeline; S3 KV-cache sampling does not use
230
- text CFG.
231
 
232
  ## Ethical Considerations:
233
 
 
149
  **Acceleration Engine:** FlashAttention (FA3/FA4); PyTorch SDPA (cuDNN) fallback on Ampere
150
  **Test Hardware:** NVIDIA A100 80GB, NVIDIA H100, NVIDIA GB200 (Blackwell)
151
 
152
+ ### Cosmos3-Nano checkpoints
153
 
154
  | Checkpoint | Training stage | Sampling |
155
  | --- | --- | --- |
156
  | [`artifixer-s1-cosmos3-nano.pt`](./artifixer-s1-cosmos3-nano.pt) | S1 bidirectional teacher, step 25,000 | Bidirectional, 50 steps, text guidance 5 |
157
  | [`artifixer-cosmos3-nano.pt`](./artifixer-cosmos3-nano.pt) | S3 DMD, EMA step 700 | Causal KV cache, 4 steps; block/window/sink = 7/21/7 latent frames |
158
 
159
+ Keep each checkpoint's `.metadata.json` sidecar beside its `.pt` file. Each
160
+ FP32 checkpoint is approximately 63.2 GB; BF16 transformer weights alone need
161
+ 31.6 GB, before the VAE, activations, and KV cache.
 
 
162
 
163
+ See the [Cosmos3-Nano inference guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/inference.md#cosmos3-nano)
164
+ for setup, downloads, and S1/S3 commands, and the
165
+ [training guide](https://github.com/nv-tlabs/ArtiFixer/blob/main/docs/training.md)
166
+ for the three-stage recipe.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
167
 
168
  ## Ethical Considerations:
169