Statuo
/

Mistral_Nemo_Instruct_EXL2_8bpw

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions

Statuo commited on Jul 18, 2024

Commit

3455ffa

·

verified ·

1 Parent(s): 7dfc482

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -16,5 +16,5 @@ tags:
 EXL2 quanting seemed to work. I ran a few tests on it and it seemed to have zero issues generating text up to 32k context size. I did not try higher than that, but uploading so folks can start testing this. Pleasantly surprised for a roleplay capacity as it seemed to latch onto character traits very well.
-[6BPW - Coming Soon](https://huggingface.co/Statuo/Mistral_Nemo_Instruct_EXL2_6bpw)
-[4BPW - Coming Soon](https://huggingface.co/)

 EXL2 quanting seemed to work. I ran a few tests on it and it seemed to have zero issues generating text up to 32k context size. I did not try higher than that, but uploading so folks can start testing this. Pleasantly surprised for a roleplay capacity as it seemed to latch onto character traits very well.
+[6BPW](https://huggingface.co/Statuo/Mistral_Nemo_Instruct_EXL2_6bpw)
+[4BPW](https://huggingface.co/Statuo/Mistral_Nemo_Instruct_EXL2_4bpw)