btbtyler09 commited on
Commit
4bcf71d
·
verified ·
1 Parent(s): a9b66b9

README: GGUF Versions row — full context per slot rule

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -293,7 +293,7 @@ serving-throughput numbers.
293
  |---|---|---|---|
294
  | shrew-ocr-preview | bf16 (this repo) | 7.5 GB | reference quality |
295
  | [shrew-ocr-preview-GPTQ-8bit](https://huggingface.co/btbtyler09/shrew-ocr-preview-GPTQ-8bit) | INT8 LM / bf16 vision | 4.9 GB | ~1.8× serving throughput, +0.25% domain perplexity; serve with `--dtype half` |
296
- | [shrew-ocr-preview-GGUF](https://huggingface.co/btbtyler09/shrew-ocr-preview-GGUF) | Q8_0 or f16 LM / f16 vision | 3.6–6.8 GB | llama.cpp; single-slot full context required (`-c 32768 -np 1`) |
297
  | [shrew-ocr-preview-lora](https://huggingface.co/btbtyler09/shrew-ocr-preview-lora) | LoRA adapter (r=256, bf16) | 2.0 GB | for composition / continued training — serve the merged variants instead |
298
 
299
  This is a preview: weights update in place under these names as the model improves. Each weight
 
293
  |---|---|---|---|
294
  | shrew-ocr-preview | bf16 (this repo) | 7.5 GB | reference quality |
295
  | [shrew-ocr-preview-GPTQ-8bit](https://huggingface.co/btbtyler09/shrew-ocr-preview-GPTQ-8bit) | INT8 LM / bf16 vision | 4.9 GB | ~1.8× serving throughput, +0.25% domain perplexity; serve with `--dtype half` |
296
+ | [shrew-ocr-preview-GGUF](https://huggingface.co/btbtyler09/shrew-ocr-preview-GGUF) | Q8_0 or f16 LM / f16 vision | 3.6–6.8 GB | llama.cpp; full context per slot required (`-c` = N × 32768) |
297
  | [shrew-ocr-preview-lora](https://huggingface.co/btbtyler09/shrew-ocr-preview-lora) | LoRA adapter (r=256, bf16) | 2.0 GB | for composition / continued training — serve the merged variants instead |
298
 
299
  This is a preview: weights update in place under these names as the model improves. Each weight