README: GGUF Versions row — full context per slot rule
Browse files
README.md
CHANGED
|
@@ -293,7 +293,7 @@ serving-throughput numbers.
|
|
| 293 |
|---|---|---|---|
|
| 294 |
| shrew-ocr-preview | bf16 (this repo) | 7.5 GB | reference quality |
|
| 295 |
| [shrew-ocr-preview-GPTQ-8bit](https://huggingface.co/btbtyler09/shrew-ocr-preview-GPTQ-8bit) | INT8 LM / bf16 vision | 4.9 GB | ~1.8× serving throughput, +0.25% domain perplexity; serve with `--dtype half` |
|
| 296 |
-
| [shrew-ocr-preview-GGUF](https://huggingface.co/btbtyler09/shrew-ocr-preview-GGUF) | Q8_0 or f16 LM / f16 vision | 3.6–6.8 GB | llama.cpp;
|
| 297 |
| [shrew-ocr-preview-lora](https://huggingface.co/btbtyler09/shrew-ocr-preview-lora) | LoRA adapter (r=256, bf16) | 2.0 GB | for composition / continued training — serve the merged variants instead |
|
| 298 |
|
| 299 |
This is a preview: weights update in place under these names as the model improves. Each weight
|
|
|
|
| 293 |
|---|---|---|---|
|
| 294 |
| shrew-ocr-preview | bf16 (this repo) | 7.5 GB | reference quality |
|
| 295 |
| [shrew-ocr-preview-GPTQ-8bit](https://huggingface.co/btbtyler09/shrew-ocr-preview-GPTQ-8bit) | INT8 LM / bf16 vision | 4.9 GB | ~1.8× serving throughput, +0.25% domain perplexity; serve with `--dtype half` |
|
| 296 |
+
| [shrew-ocr-preview-GGUF](https://huggingface.co/btbtyler09/shrew-ocr-preview-GGUF) | Q8_0 or f16 LM / f16 vision | 3.6–6.8 GB | llama.cpp; full context per slot required (`-c` = N × 32768) |
|
| 297 |
| [shrew-ocr-preview-lora](https://huggingface.co/btbtyler09/shrew-ocr-preview-lora) | LoRA adapter (r=256, bf16) | 2.0 GB | for composition / continued training — serve the merged variants instead |
|
| 298 |
|
| 299 |
This is a preview: weights update in place under these names as the model improves. Each weight
|