HIP out of memory
I have cloned the github page , and 7B huggingface repository and tried to load model.
Shouldn't it have smaller memory footprint since its a binary model ?
I am on 7600 XT ROCM on archlinux (text generation webui and comfyui and all custom models I tried has worked so far on this setup)
OutOfMemoryError: HIP out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 15.98 GiB of which 40.00 MiB is free. Of the allocated memory 15.75 GiB is allocated by PyTorch, and 1.37 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_HIP_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
What can I do about this ?