Buckets:

twanghcmut
/

backup-foundation-physics

Files

xet

twanghcmut/backup-foundation-physics / logs /cosmos_split.log

twanghcmut

8 days ago

download

raw

26.3 kB

	spec: /srv/data/foundation-physics-graph-model/outputs/cosmos_input/spec_split_masked.json
	output: /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked
	devices: 0,1
	hf_token: hf_AvnPC... (the gated downloads must use this one)
	torchrun with 2 processes (context parallel)
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766]
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766] *****************************************
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766] *****************************************
	[07-31 19:03:13\|INFO\|packages/cosmos-oss/cosmos_oss/init.py:96:_init_log_files] Log saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/console.log
	[07-31 19:03:42\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Wan2.1/vae(685afcaa-4de2-42fe-b7b9-69f7a2dee4d8)
	[07-31 19:03:42\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Predict2.5-2B --repo-type model --revision f176dc95b4a70f53ce01c4b302851595e7322b00 tokenizer.pth
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Predict2.5-2B/snapshots/f176dc95b4a70f53ce01c4b302851595e7322b00/tokenizer.pth
	[07-31 19:03:45\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Qwen/Qwen2.5-VL-7B-Instruct(7219c6c7-f878-4137-bbdb-76842ea85e70)
	[07-31 19:03:45\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	[07-31 19:03:48\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Reason1.1-7B(cb3e3ffa-7b08-4c34-822d-61c7aa31a14f)
	[07-31 19:03:48\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:04:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:04:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision b67b64abda3801a9aceddbff2bdb86126c06db74 general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:04:13\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:04:15\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/blur(ba2f44f2-c726-4fe7-949f-597069d9b91c)
	[07-31 19:04:15\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision eb5325b77d358944da58a690157dd2b8071bbf85 general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	[07-31 19:04:20\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/depth(626e6618-bfcd-4d9a-a077-1409e2ce353f)
	[07-31 19:04:20\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision dea7737ca29dd8d9086413c6dc5724b8250a0bb4 general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	[07-31 19:04:23\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/seg(5136ef49-6d8d-42e8-8abf-7dac722a304a)
	[07-31 19:04:23\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision 23057a4167b89de89a4a397fdbf3887994d115eb general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:146:__init__] Found 2 hint keys across all samples
	[07-31 19:04:27\|WARNING\|cosmos_transfer2/inference.py:148:__init__] Loading the multicontrol model. Multicontrol inference is not strictly equal to single control
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:156:__init__] Saved config to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/config.yaml
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:163:generate] Generating 1 samples: ['split_masked']
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:167:generate] [1/1] Processing sample split_masked
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:208:_generate_sample] Saved arguments to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked.json
	[07-31 19:04:27\|WARNING\|cosmos_transfer2/inference.py:239:_generate_sample] Guardrail checks on prompt are disabled
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:242:_generate_sample] Processing the following paths: {'vis': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/vis.mp4', 'vis_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/bg_mask.mp4', 'vis_mask_prompt': None, 'depth': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/depth.mp4', 'depth_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/fg_mask.mp4', 'depth_mask_prompt': None}
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:02<01:23, 2.45s/it] Generating samples: 3%\|▎ \| 1/35 [00:02<01:23, 2.46s/it] Generating samples: 6%\|▌ \| 2/35 [00:03<00:56, 1.71s/it] Generating samples: 6%\|▌ \| 2/35 [00:03<00:56, 1.71s/it] Generating samples: 9%\|▊ \| 3/35 [00:04<00:49, 1.54s/it] Generating samples: 9%\|▊ \| 3/35 [00:04<00:49, 1.54s/it] Generating samples: 11%\|█▏ \| 4/35 [00:06<00:43, 1.40s/it] Generating samples: 11%\|█▏ \| 4/35 [00:06<00:43, 1.40s/it] Generating samples: 14%\|█▍ \| 5/35 [00:07<00:39, 1.33s/it] Generating samples: 14%\|█▍ \| 5/35 [00:07<00:39, 1.33s/it] Generating samples: 17%\|█▋ \| 6/35 [00:08<00:37, 1.29s/it] Generating samples: 17%\|█▋ \| 6/35 [00:08<00:37, 1.29s/it] Generating samples: 20%\|██ \| 7/35 [00:09<00:35, 1.26s/it] Generating samples: 20%\|██ \| 7/35 [00:09<00:35, 1.26s/it] Generating samples: 23%\|██▎ \| 8/35 [00:10<00:33, 1.24s/it] Generating samples: 23%\|██▎ \| 8/35 [00:10<00:33, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:12<00:32, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:12<00:32, 1.24s/it] Generating samples: 29%\|██▊ \| 10/35 [00:13<00:30, 1.23s/it] Generating samples: 29%\|██▊ \| 10/35 [00:13<00:30, 1.23s/it] Generating samples: 31%\|███▏ \| 11/35 [00:14<00:29, 1.22s/it] Generating samples: 31%\|███▏ \| 11/35 [00:14<00:29, 1.22s/it] Generating samples: 34%\|███▍ \| 12/35 [00:15<00:28, 1.22s/it] Generating samples: 34%\|███▍ \| 12/35 [00:15<00:28, 1.22s/it] Generating samples: 37%\|███▋ \| 13/35 [00:17<00:26, 1.22s/it] Generating samples: 37%\|███▋ \| 13/35 [00:17<00:26, 1.22s/it] Generating samples: 40%\|████ \| 14/35 [00:18<00:25, 1.22s/it] Generating samples: 40%\|████ \| 14/35 [00:18<00:25, 1.22s/it] Generating samples: 43%\|████▎ \| 15/35 [00:19<00:24, 1.23s/it] Generating samples: 43%\|████▎ \| 15/35 [00:19<00:24, 1.23s/it] Generating samples: 46%\|████▌ \| 16/35 [00:20<00:23, 1.22s/it] Generating samples: 46%\|████▌ \| 16/35 [00:20<00:23, 1.22s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:21, 1.22s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:21, 1.22s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:23<00:20, 1.23s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:23<00:20, 1.23s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:24<00:19, 1.22s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:24<00:19, 1.22s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:25<00:18, 1.22s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:25<00:18, 1.22s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.23s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.23s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:28<00:15, 1.22s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:28<00:15, 1.22s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:29<00:14, 1.21s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:29<00:14, 1.21s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:30<00:13, 1.20s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:30<00:13, 1.21s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.21s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.21s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:10, 1.21s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:10, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:34<00:09, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:34<00:09, 1.21s/it] Generating samples: 80%\|████████ \| 28/35 [00:35<00:08, 1.21s/it] Generating samples: 80%\|████████ \| 28/35 [00:35<00:08, 1.22s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.21s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.21s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.22s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.22s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:04, 1.21s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:04, 1.21s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:40<00:03, 1.21s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:40<00:03, 1.21s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.22s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.22s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.21s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.21s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.21s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it]
	Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.21s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it]
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:01<00:41, 1.21s/it] Generating samples: 3%\|▎ \| 1/35 [00:01<00:41, 1.21s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.23s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.23s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:38, 1.21s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:38, 1.21s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:37, 1.20s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:37, 1.20s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.21s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.21s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:34, 1.20s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:34, 1.20s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:33, 1.20s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:33, 1.20s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:32, 1.21s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:32, 1.21s/it] Generating samples: 26%\|██▌ \| 9/35 [00:10<00:31, 1.20s/it] Generating samples: 26%\|██▌ \| 9/35 [00:10<00:31, 1.20s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.21s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.21s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.21s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.21s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:27, 1.21s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:27, 1.21s/it] Generating samples: 37%\|███▋ \| 13/35 [00:15<00:26, 1.21s/it] Generating samples: 37%\|███▋ \| 13/35 [00:15<00:26, 1.21s/it] Generating samples: 40%\|████ \| 14/35 [00:16<00:25, 1.22s/it] Generating samples: 40%\|████ \| 14/35 [00:16<00:25, 1.22s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.21s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.21s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:22, 1.21s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:22, 1.21s/it] Generating samples: 49%\|████▊ \| 17/35 [00:20<00:21, 1.21s/it] Generating samples: 49%\|████▊ \| 17/35 [00:20<00:21, 1.21s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:21<00:20, 1.20s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:21<00:20, 1.20s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:22<00:19, 1.20s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:22<00:19, 1.20s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.21s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.21s/it] Generating samples: 60%\|██████ \| 21/35 [00:25<00:16, 1.21s/it] Generating samples: 60%\|██████ \| 21/35 [00:25<00:16, 1.21s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:26<00:15, 1.21s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:26<00:15, 1.21s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:27<00:14, 1.20s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:27<00:14, 1.20s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.21s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.21s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:30<00:12, 1.20s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:30<00:12, 1.20s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:31<00:10, 1.21s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:31<00:10, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:32<00:09, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:32<00:09, 1.21s/it] Generating samples: 80%\|████████ \| 28/35 [00:33<00:08, 1.22s/it] Generating samples: 80%\|████████ \| 28/35 [00:33<00:08, 1.22s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:35<00:07, 1.22s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:35<00:07, 1.22s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:36<00:06, 1.23s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:36<00:06, 1.23s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:37<00:04, 1.23s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:37<00:04, 1.23s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:38<00:03, 1.23s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:38<00:03, 1.23s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:40<00:02, 1.24s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:40<00:02, 1.24s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:41<00:01, 1.24s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:41<00:01, 1.24s/it] Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.24s/it] Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.24s/it] Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.22s/it]
	Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.22s/it]
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:01<00:42, 1.24s/it] Generating samples: 3%\|▎ \| 1/35 [00:01<00:42, 1.24s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.24s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.24s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:39, 1.24s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:39, 1.24s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:38, 1.23s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:38, 1.23s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.23s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.23s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:35, 1.23s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:35, 1.23s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:34, 1.24s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:34, 1.24s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:33, 1.24s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:33, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:11<00:32, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:11<00:32, 1.24s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.24s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.24s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.24s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.24s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:28, 1.24s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:28, 1.24s/it] Generating samples: 37%\|███▋ \| 13/35 [00:16<00:27, 1.24s/it] Generating samples: 37%\|███▋ \| 13/35 [00:16<00:27, 1.24s/it] Generating samples: 40%\|████ \| 14/35 [00:17<00:25, 1.24s/it] Generating samples: 40%\|████ \| 14/35 [00:17<00:25, 1.24s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.24s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.24s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:23, 1.24s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:23, 1.24s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:22, 1.24s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:22, 1.24s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:22<00:21, 1.24s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:22<00:21, 1.24s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:23<00:19, 1.25s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:23<00:20, 1.25s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.25s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.25s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.25s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.25s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:27<00:16, 1.24s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:27<00:16, 1.25s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:28<00:14, 1.25s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:28<00:14, 1.25s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.24s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.24s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.24s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.24s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:11, 1.25s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:11, 1.25s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:33<00:10, 1.25s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:33<00:10, 1.25s/it] Generating samples: 80%\|████████ \| 28/35 [00:34<00:08, 1.25s/it] Generating samples: 80%\|████████ \| 28/35 [00:34<00:08, 1.25s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.25s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.25s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.25s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.25s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:05, 1.26s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:05, 1.26s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:39<00:03, 1.25s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:39<00:03, 1.25s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.25s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.25s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.25s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.25s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.24s/it]
	Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.24s/it]
	[libx264 @ 0xd3ed900] VBV maxrate unspecified, assuming CBR
	[07-31 19:07:19\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] vis control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked_control_vis.mp4
	[libx264 @ 0x27359900] VBV maxrate unspecified, assuming CBR
	[07-31 19:07:22\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] depth control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked_control_depth.mp4
	[07-31 19:07:22\|WARNING\|cosmos_transfer2/inference.py:335:_generate_sample] Guardrail checks on video are disabled
	[libx264 @ 0x3e40c900] VBV maxrate unspecified, assuming CBR
	[07-31 19:07:28\|SUCCESS\|cosmos_transfer2/inference.py:343:_generate_sample] Generated video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked.mp4

Xet Storage Details

Size:: 26.3 kB
Xet hash:: 481a334f88b6978ecb08836ab6e06cd93709bc0f17367af355c2a18bde198437

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	spec: /srv/data/foundation-physics-graph-model/outputs/cosmos_input/spec_split_masked.json
	output: /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked
	devices: 0,1
	hf_token: hf_AvnPC... (the gated downloads must use this one)
	torchrun with 2 processes (context parallel)
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766]
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766] *****************************************
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
	W0731 19:03:09.453000 3399746 torch/distributed/run.py:766] *****************************************
	[07-31 19:03:13\|INFO\|packages/cosmos-oss/cosmos_oss/init.py:96:_init_log_files] Log saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/console.log
	[07-31 19:03:42\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Wan2.1/vae(685afcaa-4de2-42fe-b7b9-69f7a2dee4d8)
	[07-31 19:03:42\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Predict2.5-2B --repo-type model --revision f176dc95b4a70f53ce01c4b302851595e7322b00 tokenizer.pth
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Predict2.5-2B/snapshots/f176dc95b4a70f53ce01c4b302851595e7322b00/tokenizer.pth
	[07-31 19:03:45\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint Qwen/Qwen2.5-VL-7B-Instruct(7219c6c7-f878-4137-bbdb-76842ea85e70)
	[07-31 19:03:45\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	[07-31 19:03:48\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Reason1.1-7B(cb3e3ffa-7b08-4c34-822d-61c7aa31a14f)
	[07-31 19:03:48\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Reason1-7B --repo-type model --revision 3210bec0495fdc7a8d3dbb8d58da5711eab4b423 --include '*'
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Reason1-7B/snapshots/3210bec0495fdc7a8d3dbb8d58da5711eab4b423
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:04:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:04:07\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision b67b64abda3801a9aceddbff2bdb86126c06db74 general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/b67b64abda3801a9aceddbff2bdb86126c06db74/general/edge/61f5694b-0ad5-4ecd-8ad7-c8545627d125_ema_bf16.pt
	[07-31 19:04:13\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/edge(61f5694b-0ad5-4ecd-8ad7-c8545627d125)
	[07-31 19:04:15\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/blur(ba2f44f2-c726-4fe7-949f-597069d9b91c)
	[07-31 19:04:15\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision eb5325b77d358944da58a690157dd2b8071bbf85 general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/eb5325b77d358944da58a690157dd2b8071bbf85/general/blur/ba2f44f2-c726-4fe7-949f-597069d9b91c_ema_bf16.pt
	[07-31 19:04:20\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/depth(626e6618-bfcd-4d9a-a077-1409e2ce353f)
	[07-31 19:04:20\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision dea7737ca29dd8d9086413c6dc5724b8250a0bb4 general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/dea7737ca29dd8d9086413c6dc5724b8250a0bb4/general/depth/626e6618-bfcd-4d9a-a077-1409e2ce353f_ema_bf16.pt
	[07-31 19:04:23\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:297:download] Downloading checkpoint nvidia/Cosmos-Transfer2.5-2B/general/seg(5136ef49-6d8d-42e8-8abf-7dac722a304a)
	[07-31 19:04:23\|INFO\|cosmos_transfer2/_src/imaginaire/utils/checkpoint_db.py:165:_hf_download] uvx 'hf>=1.3.5' download nvidia/Cosmos-Transfer2.5-2B --repo-type model --revision 23057a4167b89de89a4a397fdbf3887994d115eb general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	path=/home/quang/.cache/huggingface/hub/models--nvidia--Cosmos-Transfer2.5-2B/snapshots/23057a4167b89de89a4a397fdbf3887994d115eb/general/seg/5136ef49-6d8d-42e8-8abf-7dac722a304a_ema_bf16.pt
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:146:__init__] Found 2 hint keys across all samples
	[07-31 19:04:27\|WARNING\|cosmos_transfer2/inference.py:148:__init__] Loading the multicontrol model. Multicontrol inference is not strictly equal to single control
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:156:__init__] Saved config to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/config.yaml
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:163:generate] Generating 1 samples: ['split_masked']
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:167:generate] [1/1] Processing sample split_masked
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:208:_generate_sample] Saved arguments to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked.json
	[07-31 19:04:27\|WARNING\|cosmos_transfer2/inference.py:239:_generate_sample] Guardrail checks on prompt are disabled
	[07-31 19:04:27\|INFO\|cosmos_transfer2/inference.py:242:_generate_sample] Processing the following paths: {'vis': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/vis.mp4', 'vis_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/bg_mask.mp4', 'vis_mask_prompt': None, 'depth': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/depth.mp4', 'depth_mask': '/srv/data/foundation-physics-graph-model/outputs/cosmos_input/fg_mask.mp4', 'depth_mask_prompt': None}
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:02<01:23, 2.45s/it] Generating samples: 3%\|▎ \| 1/35 [00:02<01:23, 2.46s/it] Generating samples: 6%\|▌ \| 2/35 [00:03<00:56, 1.71s/it] Generating samples: 6%\|▌ \| 2/35 [00:03<00:56, 1.71s/it] Generating samples: 9%\|▊ \| 3/35 [00:04<00:49, 1.54s/it] Generating samples: 9%\|▊ \| 3/35 [00:04<00:49, 1.54s/it] Generating samples: 11%\|█▏ \| 4/35 [00:06<00:43, 1.40s/it] Generating samples: 11%\|█▏ \| 4/35 [00:06<00:43, 1.40s/it] Generating samples: 14%\|█▍ \| 5/35 [00:07<00:39, 1.33s/it] Generating samples: 14%\|█▍ \| 5/35 [00:07<00:39, 1.33s/it] Generating samples: 17%\|█▋ \| 6/35 [00:08<00:37, 1.29s/it] Generating samples: 17%\|█▋ \| 6/35 [00:08<00:37, 1.29s/it] Generating samples: 20%\|██ \| 7/35 [00:09<00:35, 1.26s/it] Generating samples: 20%\|██ \| 7/35 [00:09<00:35, 1.26s/it] Generating samples: 23%\|██▎ \| 8/35 [00:10<00:33, 1.24s/it] Generating samples: 23%\|██▎ \| 8/35 [00:10<00:33, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:12<00:32, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:12<00:32, 1.24s/it] Generating samples: 29%\|██▊ \| 10/35 [00:13<00:30, 1.23s/it] Generating samples: 29%\|██▊ \| 10/35 [00:13<00:30, 1.23s/it] Generating samples: 31%\|███▏ \| 11/35 [00:14<00:29, 1.22s/it] Generating samples: 31%\|███▏ \| 11/35 [00:14<00:29, 1.22s/it] Generating samples: 34%\|███▍ \| 12/35 [00:15<00:28, 1.22s/it] Generating samples: 34%\|███▍ \| 12/35 [00:15<00:28, 1.22s/it] Generating samples: 37%\|███▋ \| 13/35 [00:17<00:26, 1.22s/it] Generating samples: 37%\|███▋ \| 13/35 [00:17<00:26, 1.22s/it] Generating samples: 40%\|████ \| 14/35 [00:18<00:25, 1.22s/it] Generating samples: 40%\|████ \| 14/35 [00:18<00:25, 1.22s/it] Generating samples: 43%\|████▎ \| 15/35 [00:19<00:24, 1.23s/it] Generating samples: 43%\|████▎ \| 15/35 [00:19<00:24, 1.23s/it] Generating samples: 46%\|████▌ \| 16/35 [00:20<00:23, 1.22s/it] Generating samples: 46%\|████▌ \| 16/35 [00:20<00:23, 1.22s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:21, 1.22s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:21, 1.22s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:23<00:20, 1.23s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:23<00:20, 1.23s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:24<00:19, 1.22s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:24<00:19, 1.22s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:25<00:18, 1.22s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:25<00:18, 1.22s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.23s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.23s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:28<00:15, 1.22s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:28<00:15, 1.22s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:29<00:14, 1.21s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:29<00:14, 1.21s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:30<00:13, 1.20s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:30<00:13, 1.21s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.21s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.21s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:10, 1.21s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:10, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:34<00:09, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:34<00:09, 1.21s/it] Generating samples: 80%\|████████ \| 28/35 [00:35<00:08, 1.21s/it] Generating samples: 80%\|████████ \| 28/35 [00:35<00:08, 1.22s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.21s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.21s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.22s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.22s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:04, 1.21s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:04, 1.21s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:40<00:03, 1.21s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:40<00:03, 1.21s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.22s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.22s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.21s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.21s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.21s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it]
	Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.21s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it]
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:01<00:41, 1.21s/it] Generating samples: 3%\|▎ \| 1/35 [00:01<00:41, 1.21s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.23s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.23s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:38, 1.21s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:38, 1.21s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:37, 1.20s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:37, 1.20s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.21s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.21s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:34, 1.20s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:34, 1.20s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:33, 1.20s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:33, 1.20s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:32, 1.21s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:32, 1.21s/it] Generating samples: 26%\|██▌ \| 9/35 [00:10<00:31, 1.20s/it] Generating samples: 26%\|██▌ \| 9/35 [00:10<00:31, 1.20s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.21s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.21s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.21s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.21s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:27, 1.21s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:27, 1.21s/it] Generating samples: 37%\|███▋ \| 13/35 [00:15<00:26, 1.21s/it] Generating samples: 37%\|███▋ \| 13/35 [00:15<00:26, 1.21s/it] Generating samples: 40%\|████ \| 14/35 [00:16<00:25, 1.22s/it] Generating samples: 40%\|████ \| 14/35 [00:16<00:25, 1.22s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.21s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.21s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:22, 1.21s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:22, 1.21s/it] Generating samples: 49%\|████▊ \| 17/35 [00:20<00:21, 1.21s/it] Generating samples: 49%\|████▊ \| 17/35 [00:20<00:21, 1.21s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:21<00:20, 1.20s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:21<00:20, 1.20s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:22<00:19, 1.20s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:22<00:19, 1.20s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.21s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.21s/it] Generating samples: 60%\|██████ \| 21/35 [00:25<00:16, 1.21s/it] Generating samples: 60%\|██████ \| 21/35 [00:25<00:16, 1.21s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:26<00:15, 1.21s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:26<00:15, 1.21s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:27<00:14, 1.20s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:27<00:14, 1.20s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.21s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.21s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:30<00:12, 1.20s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:30<00:12, 1.20s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:31<00:10, 1.21s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:31<00:10, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:32<00:09, 1.21s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:32<00:09, 1.21s/it] Generating samples: 80%\|████████ \| 28/35 [00:33<00:08, 1.22s/it] Generating samples: 80%\|████████ \| 28/35 [00:33<00:08, 1.22s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:35<00:07, 1.22s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:35<00:07, 1.22s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:36<00:06, 1.23s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:36<00:06, 1.23s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:37<00:04, 1.23s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:37<00:04, 1.23s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:38<00:03, 1.23s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:38<00:03, 1.23s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:40<00:02, 1.24s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:40<00:02, 1.24s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:41<00:01, 1.24s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:41<00:01, 1.24s/it] Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.24s/it] Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.24s/it] Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.22s/it]
	Generating samples: 100%\|██████████\| 35/35 [00:42<00:00, 1.22s/it]
	Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 0%\| \| 0/35 [00:00<?, ?it/s] Generating samples: 3%\|▎ \| 1/35 [00:01<00:42, 1.24s/it] Generating samples: 3%\|▎ \| 1/35 [00:01<00:42, 1.24s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.24s/it] Generating samples: 6%\|▌ \| 2/35 [00:02<00:40, 1.24s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:39, 1.24s/it] Generating samples: 9%\|▊ \| 3/35 [00:03<00:39, 1.24s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:38, 1.23s/it] Generating samples: 11%\|█▏ \| 4/35 [00:04<00:38, 1.23s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.23s/it] Generating samples: 14%\|█▍ \| 5/35 [00:06<00:36, 1.23s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:35, 1.23s/it] Generating samples: 17%\|█▋ \| 6/35 [00:07<00:35, 1.23s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:34, 1.24s/it] Generating samples: 20%\|██ \| 7/35 [00:08<00:34, 1.24s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:33, 1.24s/it] Generating samples: 23%\|██▎ \| 8/35 [00:09<00:33, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:11<00:32, 1.24s/it] Generating samples: 26%\|██▌ \| 9/35 [00:11<00:32, 1.24s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.24s/it] Generating samples: 29%\|██▊ \| 10/35 [00:12<00:30, 1.24s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.24s/it] Generating samples: 31%\|███▏ \| 11/35 [00:13<00:29, 1.24s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:28, 1.24s/it] Generating samples: 34%\|███▍ \| 12/35 [00:14<00:28, 1.24s/it] Generating samples: 37%\|███▋ \| 13/35 [00:16<00:27, 1.24s/it] Generating samples: 37%\|███▋ \| 13/35 [00:16<00:27, 1.24s/it] Generating samples: 40%\|████ \| 14/35 [00:17<00:25, 1.24s/it] Generating samples: 40%\|████ \| 14/35 [00:17<00:25, 1.24s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.24s/it] Generating samples: 43%\|████▎ \| 15/35 [00:18<00:24, 1.24s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:23, 1.24s/it] Generating samples: 46%\|████▌ \| 16/35 [00:19<00:23, 1.24s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:22, 1.24s/it] Generating samples: 49%\|████▊ \| 17/35 [00:21<00:22, 1.24s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:22<00:21, 1.24s/it] Generating samples: 51%\|█████▏ \| 18/35 [00:22<00:21, 1.24s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:23<00:19, 1.25s/it] Generating samples: 54%\|█████▍ \| 19/35 [00:23<00:20, 1.25s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.25s/it] Generating samples: 57%\|█████▋ \| 20/35 [00:24<00:18, 1.25s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.25s/it] Generating samples: 60%\|██████ \| 21/35 [00:26<00:17, 1.25s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:27<00:16, 1.24s/it] Generating samples: 63%\|██████▎ \| 22/35 [00:27<00:16, 1.25s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:28<00:14, 1.25s/it] Generating samples: 66%\|██████▌ \| 23/35 [00:28<00:14, 1.25s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.24s/it] Generating samples: 69%\|██████▊ \| 24/35 [00:29<00:13, 1.24s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.24s/it] Generating samples: 71%\|███████▏ \| 25/35 [00:31<00:12, 1.24s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:11, 1.25s/it] Generating samples: 74%\|███████▍ \| 26/35 [00:32<00:11, 1.25s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:33<00:10, 1.25s/it] Generating samples: 77%\|███████▋ \| 27/35 [00:33<00:10, 1.25s/it] Generating samples: 80%\|████████ \| 28/35 [00:34<00:08, 1.25s/it] Generating samples: 80%\|████████ \| 28/35 [00:34<00:08, 1.25s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.25s/it] Generating samples: 83%\|████████▎ \| 29/35 [00:36<00:07, 1.25s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.25s/it] Generating samples: 86%\|████████▌ \| 30/35 [00:37<00:06, 1.25s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:05, 1.26s/it] Generating samples: 89%\|████████▊ \| 31/35 [00:38<00:05, 1.26s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:39<00:03, 1.25s/it] Generating samples: 91%\|█████████▏\| 32/35 [00:39<00:03, 1.25s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.25s/it] Generating samples: 94%\|█████████▍\| 33/35 [00:41<00:02, 1.25s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.25s/it] Generating samples: 97%\|█████████▋\| 34/35 [00:42<00:01, 1.25s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.24s/it]
	Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.25s/it] Generating samples: 100%\|██████████\| 35/35 [00:43<00:00, 1.24s/it]
	[libx264 @ 0xd3ed900] VBV maxrate unspecified, assuming CBR
	[07-31 19:07:19\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] vis control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked_control_vis.mp4
	[libx264 @ 0x27359900] VBV maxrate unspecified, assuming CBR
	[07-31 19:07:22\|INFO\|cosmos_transfer2/inference.py:311:_generate_sample] depth control video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked_control_depth.mp4
	[07-31 19:07:22\|WARNING\|cosmos_transfer2/inference.py:335:_generate_sample] Guardrail checks on video are disabled
	[libx264 @ 0x3e40c900] VBV maxrate unspecified, assuming CBR
	[07-31 19:07:28\|SUCCESS\|cosmos_transfer2/inference.py:343:_generate_sample] Generated video saved to /srv/data/foundation-physics-graph-model/outputs/cosmos_runs/split_masked/split_masked.mp4