Buckets:

twanghcmut
/

backup-foundation-physics

Files

xet

twanghcmut/backup-foundation-physics / logs /cosmos_final720.log

twanghcmut

8 days ago

download

raw

26.4 kB

	spec: /srv/data/foundation-physics-graph-model/outputs/cosmos_input/spec_split_masked_720.json
	output: /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720
	devices: 0,1
	hf_token: hf_AvnPC... (the gated downloads must use this one)
	torchrun with 2 processes (context parallel)
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766]
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766] *****************************************
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766] *****************************************
	[07-31 19:08:32\|INFO\|packages/cosmos-oss/cosmos_oss/init.py:96:_init_log_files] Log saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/console.log
	[07-31 19:09:03\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Wan2.1/vae(685afcaa-4de2-42fe-b7b9-69f7a2dee4d8)
	[07-31 19:09:03\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Predict2.5-2B --repo-type model --revision f176dc95b4a70f53ce01c4b302851595e7322b00 tokenizer.pth
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Predict2.5-2B/snapshots/f176dc95b4a70f53ce01c4b302851595e7322b00/tokenizer.pth
	[07-31 19:09:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Qwen/Qwen2.5-VL-7B-Instruct(7219c6c7-f878-4137-bbdb-76842ea85e70)
	[07-31 19:09:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	[07-31 19:09:09\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Reason1.1-7B(cb3e3ffa-7b08-4c34-822d-61c7aa31a14f)
	[07-31 19:09:09\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:09:27\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:09:27\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision b67b64abda3801a9aceddbff2bdb86126c06db74 general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:09:34\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:09:36\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/blur(ba2f44f2-c726-4fe7-949f-597069d9b91c)
	[07-31 19:09:36\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision eb5325b77d358944da58a690157dd2b8071bbf85 general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	[07-31 19:09:41\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/depth(626e6618-bfcd-4d9a-a077-1409e2ce353f)
	[07-31 19:09:41\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision dea7737ca29dd8d9086413c6dc5724b8250a0bb4 general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	[07-31 19:09:44\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/seg(5136ef49-6d8d-42e8-8abf-7dac722a304a)
	[07-31 19:09:44\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision 23057a4167b89de89a4a397fdbf3887994d115eb general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:146:__init__] Found 2 hint keys across all samples
	[07-31 19:09:49\|WARNING\|cosmos_transfer2/inference.py:148:__init__] Loading the multicontrol model. Multicontrol inference is not strictly equal to single control
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:156:__init__] Saved config to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/config.yaml
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:163:generate] Generating 1 samples: ['split_masked_720']
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:167:generate] [1/1] Processing sample split_masked_720
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:208:_generate_sample] Saved arguments to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720.json
	[07-31 19:09:49\|WARNING\|cosmos_transfer2/inference.py:239:_generate_sample] Guardrail checks on prompt are disabled
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:242:_generate_sample] Processing the following paths: {'vis': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/vis.mp4', 'vis_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/bg_mask.mp4', 'vis_mask_prompt': None, 'depth': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/depth.mp4', 'depth_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/fg_mask.mp4', 'depth_mask_prompt': None}
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:07<04:06, 7.26s/it] Generating samples: 3%\|▎ \| 1/35 [00:07<04:07, 7.27s/it] Generating samples: 6%\|▌ \| 2/35 [00:13<03:35, 6.53s/it] Generating samples: 6%\|▌ \| 2/35 [00:13<03:35, 6.52s/it] Generating samples: 9%\|▊ \| 3/35 [00:19<03:20, 6.28s/it] Generating samples: 9%\|▊ \| 3/35 [00:19<03:20, 6.28s/it] Generating samples: 11%\|█▏ \| 4/35 [00:25<03:12, 6.22s/it] Generating samples: 11%\|█▏ \| 4/35 [00:25<03:12, 6.22s/it] Generating samples: 14%\|█▍ \| 5/35 [00:31<03:04, 6.14s/it] Generating samples: 14%\|█▍ \| 5/35 [00:31<03:04, 6.14s/it] Generating samples: 17%\|█▋ \| 6/35 [00:37<02:56, 6.08s/it] Generating samples: 17%\|█▋ \| 6/35 [00:37<02:56, 6.08s/it] Generating samples: 20%\|██ \| 7/35 [00:43<02:49, 6.06s/it] Generating samples: 20%\|██ \| 7/35 [00:43<02:49, 6.06s/it] Generating samples: 23%\|██▎ \| 8/35 [00:49<02:42, 6.02s/it] Generating samples: 23%\|██▎ \| 8/35 [00:49<02:42, 6.02s/it] Generating samples: 26%\|██▌ \| 9/35 [00:55<02:36, 6.02s/it] Generating samples: 26%\|██▌ \| 9/35 [00:55<02:36, 6.02s/it] Generating samples: 29%\|██▊ \| 10/35 [01:01<02:29, 5.98s/it] Generating samples: 29%\|██▊ \| 10/35 [01:01<02:29, 5.99s/it] Generating samples: 31%\|███▏ \| 11/35 [01:07<02:25, 6.06s/it] Generating samples: 31%\|███▏ \| 11/35 [01:07<02:25, 6.06s/it] Generating samples: 34%\|███▍ \| 12/35 [01:13<02:16, 5.94s/it] Generating samples: 34%\|███▍ \| 12/35 [01:13<02:16, 5.93s/it] Generating samples: 37%\|███▋ \| 13/35 [01:18<02:09, 5.87s/it] Generating samples: 37%\|███▋ \| 13/35 [01:18<02:09, 5.87s/it] Generating samples: 40%\|████ \| 14/35 [01:24<02:02, 5.83s/it] Generating samples: 40%\|████ \| 14/35 [01:24<02:02, 5.83s/it] Generating samples: 43%\|████▎ \| 15/35 [01:30<01:56, 5.80s/it] Generating samples: 43%\|████▎ \| 15/35 [01:30<01:56, 5.81s/it] Generating samples: 46%\|████▌ \| 16/35 [01:36<01:50, 5.80s/it] Generating samples: 46%\|████▌ \| 16/35 [01:36<01:50, 5.81s/it] Generating samples: 49%\|████▊ \| 17/35 [01:41<01:44, 5.79s/it] Generating samples: 49%\|████▊ \| 17/35 [01:41<01:44, 5.79s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:47<01:38, 5.79s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:47<01:38, 5.78s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:53<01:32, 5.79s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:53<01:32, 5.79s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:59<01:26, 5.78s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:59<01:26, 5.78s/it] Generating samples: 60%\|██████ \| 21/35 [02:05<01:20, 5.78s/it] Generating samples: 60%\|██████ \| 21/35 [02:04<01:20, 5.78s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:10<01:15, 5.78s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:10<01:15, 5.78s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:16<01:09, 5.78s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:16<01:09, 5.78s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:22<01:03, 5.75s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:22<01:03, 5.75s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:28<00:57, 5.76s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:28<00:57, 5.76s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:33<00:52, 5.79s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:33<00:52, 5.79s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:39<00:46, 5.79s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:39<00:46, 5.79s/it] Generating samples: 80%\|████████ \| 28/35 [02:45<00:40, 5.79s/it] Generating samples: 80%\|████████ \| 28/35 [02:45<00:40, 5.79s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:51<00:34, 5.79s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:51<00:34, 5.80s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:57<00:29, 5.80s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:57<00:29, 5.80s/it] Generating samples: 89%\|████████▊ \| 31/35 [03:02<00:23, 5.81s/it] Generating samples: 89%\|████████▊ \| 31/35 [03:02<00:23, 5.81s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:08<00:17, 5.79s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:08<00:17, 5.79s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:14<00:11, 5.79s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:14<00:11, 5.79s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:20<00:05, 5.77s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:20<00:05, 5.77s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.81s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.81s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.89s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.89s/it]

	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:05<03:17, 5.81s/it] Generating samples: 3%\|▎ \| 1/35 [00:05<03:17, 5.81s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:09, 5.76s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:09, 5.76s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.72s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.72s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.73s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.73s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:52, 5.77s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:52, 5.77s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:47, 5.77s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:47, 5.77s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:41, 5.77s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:41, 5.77s/it] Generating samples: 23%\|██▎ \| 8/35 [00:46<02:35, 5.77s/it] Generating samples: 23%\|██▎ \| 8/35 [00:46<02:35, 5.77s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:30, 5.78s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:30, 5.78s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:24, 5.77s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:24, 5.77s/it] Generating samples: 31%\|███▏ \| 11/35 [01:03<02:18, 5.77s/it] Generating samples: 31%\|███▏ \| 11/35 [01:03<02:18, 5.77s/it] Generating samples: 34%\|███▍ \| 12/35 [01:09<02:13, 5.79s/it] Generating samples: 34%\|███▍ \| 12/35 [01:09<02:13, 5.79s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:06, 5.76s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:06, 5.76s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:01, 5.79s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:01, 5.79s/it] Generating samples: 43%\|████▎ \| 15/35 [01:26<01:56, 5.82s/it] Generating samples: 43%\|████▎ \| 15/35 [01:26<01:56, 5.82s/it] Generating samples: 46%\|████▌ \| 16/35 [01:32<01:51, 5.85s/it] Generating samples: 46%\|████▌ \| 16/35 [01:32<01:51, 5.85s/it] Generating samples: 49%\|████▊ \| 17/35 [01:38<01:45, 5.85s/it] Generating samples: 49%\|████▊ \| 17/35 [01:38<01:45, 5.85s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:44<01:39, 5.84s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:44<01:39, 5.84s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:49<01:32, 5.80s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:49<01:32, 5.80s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:55<01:27, 5.80s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:55<01:27, 5.80s/it] Generating samples: 60%\|██████ \| 21/35 [02:01<01:21, 5.80s/it] Generating samples: 60%\|██████ \| 21/35 [02:01<01:21, 5.80s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:07<01:15, 5.80s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:07<01:15, 5.80s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:13<01:09, 5.79s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:13<01:09, 5.79s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:18<01:03, 5.78s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:18<01:03, 5.78s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:24<00:57, 5.76s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:24<00:57, 5.76s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:30<00:51, 5.77s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:30<00:51, 5.77s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:36<00:46, 5.78s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:36<00:46, 5.78s/it] Generating samples: 80%\|████████ \| 28/35 [02:41<00:40, 5.77s/it] Generating samples: 80%\|████████ \| 28/35 [02:41<00:40, 5.77s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:47<00:34, 5.73s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:47<00:34, 5.73s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:53<00:28, 5.73s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:53<00:28, 5.73s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:59<00:22, 5.73s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:59<00:22, 5.73s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:04<00:17, 5.73s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:04<00:17, 5.73s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:10<00:11, 5.73s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:10<00:11, 5.73s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:16<00:05, 5.73s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:16<00:05, 5.73s/it] Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.73s/it] Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.77s/it]
	Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.73s/it] Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.77s/it]
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:05<03:14, 5.72s/it] Generating samples: 3%\|▎ \| 1/35 [00:05<03:14, 5.72s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:08, 5.70s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:08, 5.70s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.69s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.69s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.71s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.71s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:51, 5.71s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:51, 5.71s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:46, 5.73s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:46, 5.73s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:40, 5.74s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:40, 5.74s/it] Generating samples: 23%\|██▎ \| 8/35 [00:45<02:35, 5.74s/it] Generating samples: 23%\|██▎ \| 8/35 [00:45<02:35, 5.74s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:28, 5.73s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:28, 5.73s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:23, 5.73s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:23, 5.73s/it] Generating samples: 31%\|███▏ \| 11/35 [01:02<02:17, 5.72s/it] Generating samples: 31%\|███▏ \| 11/35 [01:02<02:17, 5.72s/it] Generating samples: 34%\|███▍ \| 12/35 [01:08<02:11, 5.73s/it] Generating samples: 34%\|███▍ \| 12/35 [01:08<02:11, 5.73s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:05, 5.72s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:05, 5.72s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:00, 5.74s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:00, 5.74s/it] Generating samples: 43%\|████▎ \| 15/35 [01:25<01:54, 5.72s/it] Generating samples: 43%\|████▎ \| 15/35 [01:25<01:54, 5.72s/it] Generating samples: 46%\|████▌ \| 16/35 [01:31<01:48, 5.70s/it] Generating samples: 46%\|████▌ \| 16/35 [01:31<01:48, 5.70s/it] Generating samples: 49%\|████▊ \| 17/35 [01:37<01:42, 5.70s/it] Generating samples: 49%\|████▊ \| 17/35 [01:37<01:42, 5.70s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:42<01:37, 5.71s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:42<01:37, 5.71s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:48<01:31, 5.71s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:48<01:31, 5.71s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:54<01:25, 5.72s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:54<01:25, 5.72s/it] Generating samples: 60%\|██████ \| 21/35 [02:00<01:20, 5.72s/it] Generating samples: 60%\|██████ \| 21/35 [02:00<01:20, 5.72s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:05<01:14, 5.71s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:05<01:14, 5.71s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:11<01:08, 5.72s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:11<01:08, 5.72s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:17<01:02, 5.72s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:17<01:02, 5.72s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:22<00:57, 5.70s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:22<00:57, 5.70s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:28<00:51, 5.70s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:28<00:51, 5.70s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:34<00:45, 5.69s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:34<00:45, 5.69s/it] Generating samples: 80%\|████████ \| 28/35 [02:39<00:39, 5.69s/it] Generating samples: 80%\|████████ \| 28/35 [02:39<00:39, 5.69s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:45<00:34, 5.69s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:45<00:34, 5.69s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:51<00:28, 5.69s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:51<00:28, 5.69s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:57<00:22, 5.69s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:57<00:22, 5.69s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:02<00:17, 5.68s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:02<00:17, 5.68s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:08<00:11, 5.68s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:08<00:11, 5.68s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:14<00:05, 5.68s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:14<00:05, 5.68s/it] Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.68s/it] Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.68s/it] Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.71s/it]
	Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.71s/it]
	[libx264 @ 0x1a059900] VBV maxrate unspecified, assuming CBR
	[07-31 19:23:27\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] vis control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720_control_vis.mp4
	[libx264 @ 0x16464900] VBV maxrate unspecified, assuming CBR
	[07-31 19:23:31\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] depth control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720_control_depth.mp4
	[07-31 19:23:31\|WARNING\|cosmos_transfer2/inference.py:335:_generate_sample] Guardrail checks on video are disabled
	[libx264 @ 0x27c32900] VBV maxrate unspecified, assuming CBR
	[07-31 19:23:35\|SUCCESS\|cosmos_transfer2/inference.py:343:_generate_sample] Generated video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720.mp4

Xet Storage Details

Size:: 26.4 kB
Xet hash:: 4327380dfb9c24ebfd16eedc0cd88e9c4e75b8d75589f20c82ab99cc1961e096

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	spec: /srv/data/foundation-physics-graph-model/outputs/cosmos_input/spec_split_masked_720.json
	output: /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720
	devices: 0,1
	hf_token: hf_AvnPC... (the gated downloads must use this one)
	torchrun with 2 processes (context parallel)
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766]
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766] *****************************************
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
	W0731 19:08:28.383000 3402220 torch/distributed/run.py:766] *****************************************
	[07-31 19:08:32\|INFO\|packages/cosmos-oss/cosmos_oss/init.py:96:_init_log_files] Log saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/console.log
	[07-31 19:09:03\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Wan2.1/vae(685afcaa-4de2-42fe-b7b9-69f7a2dee4d8)
	[07-31 19:09:03\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Predict2.5-2B --repo-type model --revision f176dc95b4a70f53ce01c4b302851595e7322b00 tokenizer.pth
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Predict2.5-2B/snapshots/f176dc95b4a70f53ce01c4b302851595e7322b00/tokenizer.pth
	[07-31 19:09:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Qwen/Qwen2.5-VL-7B-Instruct(7219c6c7-f878-4137-bbdb-76842ea85e70)
	[07-31 19:09:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	[07-31 19:09:09\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Reason1.1-7B(cb3e3ffa-7b08-4c34-822d-61c7aa31a14f)
	[07-31 19:09:09\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:09:27\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:09:27\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision b67b64abda3801a9aceddbff2bdb86126c06db74 general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:09:34\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:09:36\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/blur(ba2f44f2-c726-4fe7-949f-597069d9b91c)
	[07-31 19:09:36\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision eb5325b77d358944da58a690157dd2b8071bbf85 general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	[07-31 19:09:41\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/depth(626e6618-bfcd-4d9a-a077-1409e2ce353f)
	[07-31 19:09:41\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision dea7737ca29dd8d9086413c6dc5724b8250a0bb4 general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	[07-31 19:09:44\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/seg(5136ef49-6d8d-42e8-8abf-7dac722a304a)
	[07-31 19:09:44\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision 23057a4167b89de89a4a397fdbf3887994d115eb general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:146:__init__] Found 2 hint keys across all samples
	[07-31 19:09:49\|WARNING\|cosmos_transfer2/inference.py:148:__init__] Loading the multicontrol model. Multicontrol inference is not strictly equal to single control
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:156:__init__] Saved config to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/config.yaml
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:163:generate] Generating 1 samples: ['split_masked_720']
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:167:generate] [1/1] Processing sample split_masked_720
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:208:_generate_sample] Saved arguments to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720.json
	[07-31 19:09:49\|WARNING\|cosmos_transfer2/inference.py:239:_generate_sample] Guardrail checks on prompt are disabled
	[07-31 19:09:49\|INFO\|cosmos_transfer2/inference.py:242:_generate_sample] Processing the following paths: {'vis': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/vis.mp4', 'vis_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/bg_mask.mp4', 'vis_mask_prompt': None, 'depth': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/depth.mp4', 'depth_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/fg_mask.mp4', 'depth_mask_prompt': None}
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:07<04:06, 7.26s/it] Generating samples: 3%\|▎ \| 1/35 [00:07<04:07, 7.27s/it] Generating samples: 6%\|▌ \| 2/35 [00:13<03:35, 6.53s/it] Generating samples: 6%\|▌ \| 2/35 [00:13<03:35, 6.52s/it] Generating samples: 9%\|▊ \| 3/35 [00:19<03:20, 6.28s/it] Generating samples: 9%\|▊ \| 3/35 [00:19<03:20, 6.28s/it] Generating samples: 11%\|█▏ \| 4/35 [00:25<03:12, 6.22s/it] Generating samples: 11%\|█▏ \| 4/35 [00:25<03:12, 6.22s/it] Generating samples: 14%\|█▍ \| 5/35 [00:31<03:04, 6.14s/it] Generating samples: 14%\|█▍ \| 5/35 [00:31<03:04, 6.14s/it] Generating samples: 17%\|█▋ \| 6/35 [00:37<02:56, 6.08s/it] Generating samples: 17%\|█▋ \| 6/35 [00:37<02:56, 6.08s/it] Generating samples: 20%\|██ \| 7/35 [00:43<02:49, 6.06s/it] Generating samples: 20%\|██ \| 7/35 [00:43<02:49, 6.06s/it] Generating samples: 23%\|██▎ \| 8/35 [00:49<02:42, 6.02s/it] Generating samples: 23%\|██▎ \| 8/35 [00:49<02:42, 6.02s/it] Generating samples: 26%\|██▌ \| 9/35 [00:55<02:36, 6.02s/it] Generating samples: 26%\|██▌ \| 9/35 [00:55<02:36, 6.02s/it] Generating samples: 29%\|██▊ \| 10/35 [01:01<02:29, 5.98s/it] Generating samples: 29%\|██▊ \| 10/35 [01:01<02:29, 5.99s/it] Generating samples: 31%\|███▏ \| 11/35 [01:07<02:25, 6.06s/it] Generating samples: 31%\|███▏ \| 11/35 [01:07<02:25, 6.06s/it] Generating samples: 34%\|███▍ \| 12/35 [01:13<02:16, 5.94s/it] Generating samples: 34%\|███▍ \| 12/35 [01:13<02:16, 5.93s/it] Generating samples: 37%\|███▋ \| 13/35 [01:18<02:09, 5.87s/it] Generating samples: 37%\|███▋ \| 13/35 [01:18<02:09, 5.87s/it] Generating samples: 40%\|████ \| 14/35 [01:24<02:02, 5.83s/it] Generating samples: 40%\|████ \| 14/35 [01:24<02:02, 5.83s/it] Generating samples: 43%\|████▎ \| 15/35 [01:30<01:56, 5.80s/it] Generating samples: 43%\|████▎ \| 15/35 [01:30<01:56, 5.81s/it] Generating samples: 46%\|████▌ \| 16/35 [01:36<01:50, 5.80s/it] Generating samples: 46%\|████▌ \| 16/35 [01:36<01:50, 5.81s/it] Generating samples: 49%\|████▊ \| 17/35 [01:41<01:44, 5.79s/it] Generating samples: 49%\|████▊ \| 17/35 [01:41<01:44, 5.79s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:47<01:38, 5.79s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:47<01:38, 5.78s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:53<01:32, 5.79s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:53<01:32, 5.79s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:59<01:26, 5.78s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:59<01:26, 5.78s/it] Generating samples: 60%\|██████ \| 21/35 [02:05<01:20, 5.78s/it] Generating samples: 60%\|██████ \| 21/35 [02:04<01:20, 5.78s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:10<01:15, 5.78s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:10<01:15, 5.78s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:16<01:09, 5.78s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:16<01:09, 5.78s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:22<01:03, 5.75s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:22<01:03, 5.75s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:28<00:57, 5.76s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:28<00:57, 5.76s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:33<00:52, 5.79s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:33<00:52, 5.79s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:39<00:46, 5.79s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:39<00:46, 5.79s/it] Generating samples: 80%\|████████ \| 28/35 [02:45<00:40, 5.79s/it] Generating samples: 80%\|████████ \| 28/35 [02:45<00:40, 5.79s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:51<00:34, 5.79s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:51<00:34, 5.80s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:57<00:29, 5.80s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:57<00:29, 5.80s/it] Generating samples: 89%\|████████▊ \| 31/35 [03:02<00:23, 5.81s/it] Generating samples: 89%\|████████▊ \| 31/35 [03:02<00:23, 5.81s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:08<00:17, 5.79s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:08<00:17, 5.79s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:14<00:11, 5.79s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:14<00:11, 5.79s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:20<00:05, 5.77s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:20<00:05, 5.77s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.81s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.81s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.89s/it] Generating samples: 100%\|██████████\| 35/35 [03:26<00:00, 5.89s/it]

	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:05<03:17, 5.81s/it] Generating samples: 3%\|▎ \| 1/35 [00:05<03:17, 5.81s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:09, 5.76s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:09, 5.76s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.72s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.72s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.73s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.73s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:52, 5.77s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:52, 5.77s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:47, 5.77s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:47, 5.77s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:41, 5.77s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:41, 5.77s/it] Generating samples: 23%\|██▎ \| 8/35 [00:46<02:35, 5.77s/it] Generating samples: 23%\|██▎ \| 8/35 [00:46<02:35, 5.77s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:30, 5.78s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:30, 5.78s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:24, 5.77s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:24, 5.77s/it] Generating samples: 31%\|███▏ \| 11/35 [01:03<02:18, 5.77s/it] Generating samples: 31%\|███▏ \| 11/35 [01:03<02:18, 5.77s/it] Generating samples: 34%\|███▍ \| 12/35 [01:09<02:13, 5.79s/it] Generating samples: 34%\|███▍ \| 12/35 [01:09<02:13, 5.79s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:06, 5.76s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:06, 5.76s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:01, 5.79s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:01, 5.79s/it] Generating samples: 43%\|████▎ \| 15/35 [01:26<01:56, 5.82s/it] Generating samples: 43%\|████▎ \| 15/35 [01:26<01:56, 5.82s/it] Generating samples: 46%\|████▌ \| 16/35 [01:32<01:51, 5.85s/it] Generating samples: 46%\|████▌ \| 16/35 [01:32<01:51, 5.85s/it] Generating samples: 49%\|████▊ \| 17/35 [01:38<01:45, 5.85s/it] Generating samples: 49%\|████▊ \| 17/35 [01:38<01:45, 5.85s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:44<01:39, 5.84s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:44<01:39, 5.84s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:49<01:32, 5.80s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:49<01:32, 5.80s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:55<01:27, 5.80s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:55<01:27, 5.80s/it] Generating samples: 60%\|██████ \| 21/35 [02:01<01:21, 5.80s/it] Generating samples: 60%\|██████ \| 21/35 [02:01<01:21, 5.80s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:07<01:15, 5.80s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:07<01:15, 5.80s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:13<01:09, 5.79s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:13<01:09, 5.79s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:18<01:03, 5.78s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:18<01:03, 5.78s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:24<00:57, 5.76s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:24<00:57, 5.76s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:30<00:51, 5.77s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:30<00:51, 5.77s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:36<00:46, 5.78s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:36<00:46, 5.78s/it] Generating samples: 80%\|████████ \| 28/35 [02:41<00:40, 5.77s/it] Generating samples: 80%\|████████ \| 28/35 [02:41<00:40, 5.77s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:47<00:34, 5.73s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:47<00:34, 5.73s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:53<00:28, 5.73s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:53<00:28, 5.73s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:59<00:22, 5.73s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:59<00:22, 5.73s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:04<00:17, 5.73s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:04<00:17, 5.73s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:10<00:11, 5.73s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:10<00:11, 5.73s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:16<00:05, 5.73s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:16<00:05, 5.73s/it] Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.73s/it] Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.77s/it]
	Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.73s/it] Generating samples: 100%\|██████████\| 35/35 [03:22<00:00, 5.77s/it]
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:05<03:14, 5.72s/it] Generating samples: 3%\|▎ \| 1/35 [00:05<03:14, 5.72s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:08, 5.70s/it] Generating samples: 6%\|▌ \| 2/35 [00:11<03:08, 5.70s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.69s/it] Generating samples: 9%\|▊ \| 3/35 [00:17<03:02, 5.69s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.71s/it] Generating samples: 11%\|█▏ \| 4/35 [00:22<02:57, 5.71s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:51, 5.71s/it] Generating samples: 14%\|█▍ \| 5/35 [00:28<02:51, 5.71s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:46, 5.73s/it] Generating samples: 17%\|█▋ \| 6/35 [00:34<02:46, 5.73s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:40, 5.74s/it] Generating samples: 20%\|██ \| 7/35 [00:40<02:40, 5.74s/it] Generating samples: 23%\|██▎ \| 8/35 [00:45<02:35, 5.74s/it] Generating samples: 23%\|██▎ \| 8/35 [00:45<02:35, 5.74s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:28, 5.73s/it] Generating samples: 26%\|██▌ \| 9/35 [00:51<02:28, 5.73s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:23, 5.73s/it] Generating samples: 29%\|██▊ \| 10/35 [00:57<02:23, 5.73s/it] Generating samples: 31%\|███▏ \| 11/35 [01:02<02:17, 5.72s/it] Generating samples: 31%\|███▏ \| 11/35 [01:02<02:17, 5.72s/it] Generating samples: 34%\|███▍ \| 12/35 [01:08<02:11, 5.73s/it] Generating samples: 34%\|███▍ \| 12/35 [01:08<02:11, 5.73s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:05, 5.72s/it] Generating samples: 37%\|███▋ \| 13/35 [01:14<02:05, 5.72s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:00, 5.74s/it] Generating samples: 40%\|████ \| 14/35 [01:20<02:00, 5.74s/it] Generating samples: 43%\|████▎ \| 15/35 [01:25<01:54, 5.72s/it] Generating samples: 43%\|████▎ \| 15/35 [01:25<01:54, 5.72s/it] Generating samples: 46%\|████▌ \| 16/35 [01:31<01:48, 5.70s/it] Generating samples: 46%\|████▌ \| 16/35 [01:31<01:48, 5.70s/it] Generating samples: 49%\|████▊ \| 17/35 [01:37<01:42, 5.70s/it] Generating samples: 49%\|████▊ \| 17/35 [01:37<01:42, 5.70s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:42<01:37, 5.71s/it] Generating samples: 51%\|█████▏ \| 18/35 [01:42<01:37, 5.71s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:48<01:31, 5.71s/it] Generating samples: 54%\|█████▍ \| 19/35 [01:48<01:31, 5.71s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:54<01:25, 5.72s/it] Generating samples: 57%\|█████▋ \| 20/35 [01:54<01:25, 5.72s/it] Generating samples: 60%\|██████ \| 21/35 [02:00<01:20, 5.72s/it] Generating samples: 60%\|██████ \| 21/35 [02:00<01:20, 5.72s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:05<01:14, 5.71s/it] Generating samples: 63%\|██████▎ \| 22/35 [02:05<01:14, 5.71s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:11<01:08, 5.72s/it] Generating samples: 66%\|██████▌ \| 23/35 [02:11<01:08, 5.72s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:17<01:02, 5.72s/it] Generating samples: 69%\|██████▊ \| 24/35 [02:17<01:02, 5.72s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:22<00:57, 5.70s/it] Generating samples: 71%\|███████▏ \| 25/35 [02:22<00:57, 5.70s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:28<00:51, 5.70s/it] Generating samples: 74%\|███████▍ \| 26/35 [02:28<00:51, 5.70s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:34<00:45, 5.69s/it] Generating samples: 77%\|███████▋ \| 27/35 [02:34<00:45, 5.69s/it] Generating samples: 80%\|████████ \| 28/35 [02:39<00:39, 5.69s/it] Generating samples: 80%\|████████ \| 28/35 [02:39<00:39, 5.69s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:45<00:34, 5.69s/it] Generating samples: 83%\|████████▎ \| 29/35 [02:45<00:34, 5.69s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:51<00:28, 5.69s/it] Generating samples: 86%\|████████▌ \| 30/35 [02:51<00:28, 5.69s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:57<00:22, 5.69s/it] Generating samples: 89%\|████████▊ \| 31/35 [02:57<00:22, 5.69s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:02<00:17, 5.68s/it] Generating samples: 91%\|█████████▏\| 32/35 [03:02<00:17, 5.68s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:08<00:11, 5.68s/it] Generating samples: 94%\|█████████▍\| 33/35 [03:08<00:11, 5.68s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:14<00:05, 5.68s/it] Generating samples: 97%\|█████████▋\| 34/35 [03:14<00:05, 5.68s/it] Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.68s/it] Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.68s/it] Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.71s/it]
	Generating samples: 100%\|██████████\| 35/35 [03:19<00:00, 5.71s/it]
	[libx264 @ 0x1a059900] VBV maxrate unspecified, assuming CBR
	[07-31 19:23:27\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] vis control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720_control_vis.mp4
	[libx264 @ 0x16464900] VBV maxrate unspecified, assuming CBR
	[07-31 19:23:31\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] depth control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720_control_depth.mp4
	[07-31 19:23:31\|WARNING\|cosmos_transfer2/inference.py:335:_generate_sample] Guardrail checks on video are disabled
	[libx264 @ 0x27c32900] VBV maxrate unspecified, assuming CBR
	[07-31 19:23:35\|SUCCESS\|cosmos_transfer2/inference.py:343:_generate_sample] Generated video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/FINAL_split_masked_720/split_masked_720.mp4