***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. Warning: The cache directory for DeepSpeed Triton autotune, /mnt/data/vmo-ai-task/anhdh35/.cache/triton, appears to be on an NFS system. While this is generally acceptable, if you experience slowdowns or hanging when DeepSpeed exits, it is recommended to set the TRITON_CACHE_DIR environment variable to a non-NFS path. Warning: The cache directory for DeepSpeed Triton autotune, /mnt/data/vmo-ai-task/anhdh35/.cache/triton, appears to be on an NFS system. While this is generally acceptable, if you experience slowdowns or hanging when DeepSpeed exits, it is recommended to set the TRITON_CACHE_DIR environment variable to a non-NFS path.Warning: The cache directory for DeepSpeed Triton autotune, /mnt/data/vmo-ai-task/anhdh35/.cache/triton, appears to be on an NFS system. While this is generally acceptable, if you experience slowdowns or hanging when DeepSpeed exits, it is recommended to set the TRITON_CACHE_DIR environment variable to a non-NFS path. Warning: The cache directory for DeepSpeed Triton autotune, /mnt/data/vmo-ai-task/anhdh35/.cache/triton, appears to be on an NFS system. While this is generally acceptable, if you experience slowdowns or hanging when DeepSpeed exits, it is recommended to set the TRITON_CACHE_DIR environment variable to a non-NFS path. NCCL version 2.27.3+cuda12.9 Loading weights: 0%| | 0/723 [00:00