source/PaddleOCR/log/workerlog.2

which: no ccache in (/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/bin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/opt/clmgr/sbin:/opt/clmgr/bin:/opt/sgi/sbin:/opt/sgi/bin:/usr/share/Modules/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/c3/bin:/opt/pbs/2026.0.0/bin)
/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/utils/cpp_extension/extension_utils.py:711: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
  warnings.warn(warning_message)
Traceback (most recent call last):
  File "/home/ee/phd/eez218227/koshur-ocrv6_offline/PaddleOCR/tools/train.py", line 31, in <module>
    from ppocr.data import build_dataloader, set_signal_handlers
  File "/home/ee/phd/eez218227/koshur-ocrv6_offline/PaddleOCR/ppocr/data/__init__.py", line 23, in <module>
    import skimage
ModuleNotFoundError: No module named 'skimage'
which: no ccache in (/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/bin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/opt/clmgr/sbin:/opt/clmgr/bin:/opt/sgi/sbin:/opt/sgi/bin:/usr/share/Modules/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/c3/bin:/opt/pbs/2026.0.0/bin)
/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/utils/cpp_extension/extension_utils.py:711: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
  warnings.warn(warning_message)
Traceback (most recent call last):
  File "/home/ee/phd/eez218227/koshur-ocrv6_offline/PaddleOCR/tools/train.py", line 31, in <module>
    from ppocr.data import build_dataloader, set_signal_handlers
  File "/home/ee/phd/eez218227/koshur-ocrv6_offline/PaddleOCR/ppocr/data/__init__.py", line 35, in <module>
    from ppocr.data.imaug import transform, create_operators
  File "/home/ee/phd/eez218227/koshur-ocrv6_offline/PaddleOCR/ppocr/data/imaug/__init__.py", line 19, in <module>
    from .iaa_augment import IaaAugment
  File "/home/ee/phd/eez218227/koshur-ocrv6_offline/PaddleOCR/ppocr/data/imaug/iaa_augment.py", line 24, in <module>
    import albumentations as A
ModuleNotFoundError: No module named 'albumentations'
which: no ccache in (/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/bin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/opt/clmgr/sbin:/opt/clmgr/bin:/opt/sgi/sbin:/opt/sgi/bin:/usr/share/Modules/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/c3/bin:/opt/pbs/2026.0.0/bin)
/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/utils/cpp_extension/extension_utils.py:711: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
  warnings.warn(warning_message)
Skipping import of the encryption module.
======================= Modified FLAGS detected =======================
FLAGS(name='FLAGS_cusolver_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/cusolver/lib', default_value='')
FLAGS(name='FLAGS_nccl_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/nccl/lib', default_value='')
FLAGS(name='FLAGS_nvidia_package_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia', default_value='')
FLAGS(name='FLAGS_cublas_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/cublas/lib', default_value='')
FLAGS(name='FLAGS_cupti_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/cuda_cupti/lib', default_value='')
FLAGS(name='FLAGS_curand_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/curand/lib', default_value='')
FLAGS(name='FLAGS_selected_gpus', current_value='2', default_value='')
FLAGS(name='FLAGS_enable_pir_in_executor', current_value=True, default_value=False)
FLAGS(name='FLAGS_cudnn_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/cudnn/lib', default_value='')
FLAGS(name='FLAGS_cusparse_dir', current_value='/home/ee/phd/eez218227/koshur-ocrv6_offline/.venv/lib64/python3.11/site-packages/paddle/../nvidia/cusparse/lib', default_value='')
FLAGS(name='FLAGS_flagcx_dir', current_value='/build/lib', default_value='')
=======================================================================
I0713 04:18:41.062602 1291803 tcp_utils.cc:134] Successfully connected to 172.20.105.23:59850
I0713 04:18:41.148394 1291803 process_group_nccl.cc:151] ProcessGroupNCCL pg_timeout_ 1800000
I0713 04:18:41.148468 1291803 process_group_nccl.cc:152] ProcessGroupNCCL nccl_comm_init_option_ 0
W0713 04:18:42.934263 1291803 gpu_resources.cc:119] Please NOTE: device: 2, GPU Compute Capability: 8.0, Driver API Version: 13.0, Runtime API Version: 11.8
W0713 04:18:43.254746 1291803 gpu_resources.cc:164] device: 2, cuDNN Version: 8.9.
W0713 04:18:54.258445 1291803 gpu_resources.cc:306] WARNING: device: 2. The installed Paddle is compiled with CUDNN 8.9, but CUDNN version in your machine is 8.9, which may cause serious incompatible bug. Please recompile or reinstall Paddle with compatible CUDNN version.
I0713 20:52:13.257655 1291803 process_group_nccl.cc:159] ProcessGroupNCCL destruct