| Loading checkpoint shards: 0%| | 0/5 [00:00<?, ?it/s] Loading checkpoint shards: 20%|██████████████ | 1/5 [00:03<00:14, 3.67s/it] Loading checkpoint shards: 40%|████████████████████████████ | 2/5 [00:08<00:12, 4.23s/it] Loading checkpoint shards: 60%|██████████████████████████████████████████ | 3/5 [00:12<00:08, 4.41s/it] Loading checkpoint shards: 80%|████████████████████████████████████████████████████████ | 4/5 [00:17<00:04, 4.47s/it] Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████| 5/5 [00:20<00:00, 4.10s/it] Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████| 5/5 [00:20<00:00, 4.19s/it] | |
| [2025-10-11 03:41:18,226] [WARNING] [py.warnings._showwarnmsg:110] [PID:15466] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:680: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . | |
| warnings.warn( | |
| [2025-10-11 04:10:30,427] [WARNING] [py.warnings._showwarnmsg:110] [PID:15466] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:680: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . | |
| warnings.warn( | |
| [2025-10-11 04:39:26,571] [WARNING] [py.warnings._showwarnmsg:110] [PID:15466] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:680: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . | |
| warnings.warn( | |
| [2025-10-11 05:06:40,415] [WARNING] [py.warnings._showwarnmsg:110] [PID:15466] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:680: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . | |
| warnings.warn( | |