mradermacher

/

model_requests

like 132

I do not want F32 for this model since its 11b.

It's queued! :D
This model failed in the past so it will probably fail again. The only reason I even try it is because llmc why doesn't really give me any good reason why it failed.

You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#dipper-paraphraser-xxl-GGUF for quants to appear.

oh, what did it say? i get this: "FileNotFoundError: File not found: dipper-paraphraser-xxl/spiece.model"

Should I submit a new request or just post the link in here so we have a paper trail of whats going on?

I went ahead and cloned their repo and added spiece.model to the root so hopefully that works. Since its a stand alone model hopefully it has all the right stuff in place for the conversion. I don't mind trying to figure this out.

https://huggingface.co/dylrob1/dipper-paraphraser-xxl-fixed

It's queued! :D

You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#dipper-paraphraser-xxl-fixed-GGUF for quants to appear.

Should I submit a new request or just post the link in here so we have a paper trail of whats going on?

No here is perfectly fine.

I went ahead and cloned their repo and added spiece.model to the root so hopefully that works. Since its a stand alone model hopefully it has all the right stuff in place for the conversion. I don't mind trying to figure this out.

Where did you get spiece.model or how you generated it? For the next time instead of reuploading the entire thing you can just use https://huggingface.co/spaces/huggingface-projects/repo_duplicator to user the HuggingFace API to clone it and then just upload the missing file.

oh, what did it say? i get this: "FileNotFoundError: File not found: dipper-paraphraser-xxl/spiece.model"

Even now llmc why dipper-paraphraser-xxl is still useless. It only tells two what datatype it converted each tensor and then stops in the middle of doing so. Not really sure why it does so:

Now it failed with this error. That is super strange. Seams like at least model-00001-of-00005.safetensors you uploaded are now corrupted:

INFO:hf-to-gguf:Loading model: dipper-paraphraser-xxl-fixed
INFO:hf-to-gguf:Model architecture: T5ForConditionalGeneration
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00005.safetensors'
Traceback (most recent call last):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 7871, in <module>
    main()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 7865, in main
    model_instance.write()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 410, in write
    self.prepare_tensors()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 259, in prepare_tensors
    for name, data_torch in chain(self.generate_extra_tensors(), self.get_tensors()):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 178, in get_tensors
    ctx = cast(ContextManager[Any], safe_open(self.dir_model / part_name, framework="pt", device="cpu"))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge
job finished, status 1
job-done<0 dipper-paraphraser-xxl-fixed noquant 1>

@dylrob1 Yes it indeed is. They are all just 135 Bytes as you can see under https://huggingface.co/dylrob1/dipper-paraphraser-xxl-fixed/blob/main/model-00001-of-00005.safetensors - maybe try again but this time use https://huggingface.co/spaces/huggingface-projects/repo_duplicator instead of trying to reupload the entire model.

I will do that! thanks for the info

@nicoboss the output you get looks as if your terminal isn't fast enough and simply drops data - llmc can print a lot of data. what happens when you divert output to a file and use tail on that? also, can you try without putty? putty has a lot of issues like that.

If could even be whatever you use to switch from your login-session session to the nico1 vm, assuming you don't log in directly. In any case, I wouldn't trust it if it simply corrupts data like that (it will likely also drop data in the middle of output then) - all that llmc does is write text to the tty, as fast as it allows it, and this will likely happen more, as llmc output is copious :)

I have it up and running but not in gguf format so if it turns out to be a good model I will come back to this.