Hugging Face's logo

mradermacher
/
model_requests

English
Model card Files
xet
Community
1759

zenlm gguf models pls 🫶

#1414
by Netsnake - opened
Discussion
Netsnake

zenlm/zen-nano
zenlm/zen-nano-thinking-4bit
zenlm/zen-nano-instruct-4bit
nicoboss

They are all queued! :D
Quantized models cannot be converted into a GGUF but those might be in BF16 according to the HuggingFace model viewer so let's give them a try.
There will be a slight delay in weighted/imatrix quants due to it being behind the massive Hermes-4-405B.

You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary pages under the following URLs for quants to appear.
nicoboss

Unfortionately all your requested models failed but for diffrent reasons:

zen-nano

INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model.safetensors'
Traceback (most recent call last):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 9178, in <module>
    main()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 9172, in main
    model_instance.write()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 439, in write
    self.prepare_tensors()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 300, in prepare_tensors
    for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 3061, in modify_tensors
    yield from super().modify_tensors(data_torch, name, bid)
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 268, in modify_tensors
    return [(self.map_tensor_name(name), data_torch)]
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 259, in map_tensor_name
    raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.embed_tokens.biases'
job finished, status 1
job-done<0 zen-nano noquant 1>

zen-nano-instruct-4bit

INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model.safetensors'
Traceback (most recent call last):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 9178, in <module>
    main()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 9172, in main
    model_instance.write()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 439, in write
    self.prepare_tensors()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 300, in prepare_tensors
    for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 3061, in modify_tensors
    yield from super().modify_tensors(data_torch, name, bid)
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 268, in modify_tensors
    return [(self.map_tensor_name(name), data_torch)]
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 259, in map_tensor_name
    raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.embed_tokens.biases'
job finished, status 1
job-done<0 zen-nano-instruct-4bit noquant 1>

zen-nano-thinking-4bit:

INFO:hf-to-gguf:blk.35.attn_norm.weight,   torch.bfloat16 --> F32, shape = {2560}
INFO:hf-to-gguf:blk.35.ffn_down.weight,    torch.bfloat16 --> BF16, shape = {9728, 2560}
INFO:hf-to-gguf:blk.35.ffn_gate.weight,    torch.bfloat16 --> BF16, shape = {2560, 9728}
INFO:hf-to-gguf:blk.35.ffn_up.weight,      torch.bfloat16 --> BF16, shape = {2560, 9728}
INFO:hf-to-gguf:blk.35.ffn_norm.weight,    torch.bfloat16 --> F32, shape = {2560}
INFO:hf-to-gguf:blk.35.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.35.attn_k.weight,      torch.bfloat16 --> BF16, shape = {2560, 1024}
INFO:hf-to-gguf:blk.35.attn_output.weight, torch.bfloat16 --> BF16, shape = {4096, 2560}
INFO:hf-to-gguf:blk.35.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.35.attn_q.weight,      torch.bfloat16 --> BF16, shape = {2560, 4096}
INFO:hf-to-gguf:blk.35.attn_v.weight,      torch.bfloat16 --> BF16, shape = {2560, 1024}
INFO:hf-to-gguf:output_norm.weight,        torch.bfloat16 --> F32, shape = {2560}
INFO:hf-to-gguf:gguf: loading model part 'model.safetensors'
Traceback (most recent call last):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 9178, in <module>
    main()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 9172, in main
    model_instance.write()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 439, in write
    self.prepare_tensors()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 300, in prepare_tensors
    for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 3061, in modify_tensors
    yield from super().modify_tensors(data_torch, name, bid)
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 268, in modify_tensors
    return [(self.map_tensor_name(name), data_torch)]
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 259, in map_tensor_name
    raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.embed_tokens.biases'
job finished, status 1
job-done<0 zen-nano-thinking-4bit noquant 1>
nicoboss

@Netsnake On the bright side I think we already did them. At least we did https://huggingface.co/zenlm/zen-nano-instruct, https://huggingface.co/zenlm/zen-nano-4b-thinking and https://huggingface.co/zenlm/zen-nano-4b-instruct - no idea how they differ from the ones you requested:
Netsnake

thank you so mutch for ur efforts 🫶

Sign up or log in to comment