runtime error

Exit code: 1. Reason: /usr/local/lib/python3.10/site-packages/cupy/_environment.py:374: UserWarning: CuPy failed to preload library (/usr/local/lib/python3.10/site-packages/cutensor/lib/libcutensor.so.2): OSError (libcublasLt.so.12: cannot open shared object file: No such file or directory) warnings.warn(msg) /usr/local/lib/python3.10/site-packages/cupy/_environment.py:488: UserWarning: cudnn library could not be loaded. Reason: ImportError (libcudnn.so.8: cannot open shared object file: No such file or directory) You can install the library by: $ python -m cupyx.tools.install_library --library cudnn --cuda 12.x warnings.warn(f''' checkpoint.pkl: 0%| | 0.00/709M [00:00<?, ?B/s] checkpoint.pkl: 0%| | 393k/709M [00:01<39:20, 300kB/s] checkpoint.pkl: 15%|█▍ | 106M/709M [00:02<00:11, 51.8MB/s] checkpoint.pkl: 100%|██████████| 709M/709M [00:03<00:00, 235MB/s] Traceback (most recent call last): File "/app/app.py", line 32, in <module> model.load_state_dict(checkpoint["model"]) File "/usr/local/lib/python3.10/site-packages/neurograd/nn/module.py", line 119, in load_state_dict raise ValueError(f"Shape mismatch for '{name}': {arr.shape} vs {b.data.shape}") ValueError: Shape mismatch for 'layer_9.batch_norm.running_mean': (1, 4096) vs (1, 4096, 1, 1)

Container logs:

Fetching error logs...