Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,7 @@ pinned: true
|
|
| 10 |
These are my own quantizations (updated almost daily).
|
| 11 |
The difference with normal quantizations is that I quantize the output and embed tensors to f16.
|
| 12 |
and the other tensors to 15_k,q6_k or q8_0.
|
| 13 |
-
This creates models that are little or not degraded at all and have a smaller size.
|
| 14 |
They run at about 3-6 t/sec on CPU only using llama.cpp
|
| 15 |
And obviously faster on computers with potent GPUs
|
| 16 |
|
|
|
|
| 10 |
These are my own quantizations (updated almost daily).
|
| 11 |
The difference with normal quantizations is that I quantize the output and embed tensors to f16.
|
| 12 |
and the other tensors to 15_k,q6_k or q8_0.
|
| 13 |
+
This creates models that are little or not degraded at all and have a smaller size.
|
| 14 |
They run at about 3-6 t/sec on CPU only using llama.cpp
|
| 15 |
And obviously faster on computers with potent GPUs
|
| 16 |
|