TheDrummer/Fallen-Command-A-111B-v1
It's queued! :D
You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#/Fallen-Command-A-111B-v1-GGUF for quants to appear.
Cheers, i'll be waiting
Strange it was stuck showing this for 2 hours:
-2000 223 si Fallen-Command-A-111B-v1 run/hfd 23% of 56 files
And then imediately went to noquant and failed with this:
INFO:hf-to-gguf:Loading model: Fallen-Command-A-111B-v1
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00049.safetensors'
INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {12288, 256000}
I will let it redownload as this looks like a download issue.
Ah and now it is out of budget:
-2000 223 si Fallen-Command-A-111B-v1 budget/hfd/245G
Well I was able to "fix" the budget issue using touch Fallen-Command-A-111B-v1.nobudget - let's hope it won't run out of storage but should be fine.
Still the same issue and hash of downloaded files match:
bash-5.2$ sha256sum model-00001-of-00049.safetensors
a8b510a976376c61d1387063aec33104a962b451416d69d901089758bb00d9e8 model-00001-of-00049.safetensors
bash-5.2$ sha256sum model-00002-of-00049.safetensors
2ee4d6108410f26afb7f9bf191c775a1c2d1802f42ac09dbe3dd705e92ff33bc model-00002-of-00049.safetensors
Something is really strange about this error. First the model author made its own quants under https://huggingface.co/TheDrummer/Fallen-Command-A-111B-v1-GGUF so we know the model is llama.cpp compatible and second there is no llama.cpp error and 3rd something with the download process seamed very strange as it basically jumped from 23% to completed.
Strange it was stuck showing this for 2 hours:
That's not usually strange on rich, which sometimes gets as much as a few MBps download speed.
Something is really strange about this error.
Yeah, it's the oom killer. That sometimes happens on rich1 randomly, but this time, it seems a repeatable OOM. Let's try it out on nico1 which, unfortunately, is a bit underbudgeted as a result.
Seems to get past this on nico1 - strangely enough, top shows only 5GB rss and 16GB vm, at least right now.