mudler commited on
Commit
4e50db5
·
verified ·
1 Parent(s): 85b49ee

docs: replace em/en dashes with hyphens

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -15,23 +15,23 @@ pipeline_tag: object-detection
15
  library_name: gguf
16
  ---
17
 
18
- # locate-anything.cpp GGUF
19
 
20
  GGUF builds of [`nvidia/LocateAnything-3B`](https://huggingface.co/nvidia/LocateAnything-3B)
21
- for **[locate-anything.cpp](https://github.com/mudler/locate-anything.cpp)** a C++/ggml
22
  inference engine for open-vocabulary detection / visual grounding, no Python at inference time.
23
 
24
  **Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team.**
25
 
26
  The detections are the same as the official PyTorch implementation (the engine is
27
- parity-gated against it), and it runs faster on CPU and GPU.
28
 
29
  ## Files
30
 
31
  | File | Bits (LM) | Size | Notes |
32
  | ---- | --------- | ---- | ----- |
33
  | `locate-anything-f16.gguf` | f16 | ~9.2 GB | LM matmuls in f16, everything else f32 |
34
- | `locate-anything-q8_0.gguf` | q8_0 | ~6.3 GB | near-lossless; **box-identical** to f32 recommended |
35
  | `locate-anything-q6_k.gguf` | q6_k | ~5.5 GB | box-identical to f32 |
36
  | `locate-anything-q5_k.gguf` | q5_k | ~5.1 GB | sub-pixel box drift |
37
  | `locate-anything-q4_k.gguf` | q4_k | ~4.7 GB | smallest; sub-pixel box drift |
@@ -65,7 +65,7 @@ through q6_k:
65
 
66
  Run against the official model exactly as its model card documents (bf16), greedily, on one
67
  GB10 GPU. Precision-matched (our **f16** vs its bf16) ours is **~1.7×** faster; the
68
- recommended **q8_0** build (box-identical) is **~1.92.1×**:
69
 
70
  ![GB10 GPU speedup vs official bf16](gpu_speedup.png)
71
 
@@ -73,7 +73,7 @@ recommended **q8_0** build (box-identical) is **~1.9–2.1×**:
73
 
74
  Only the Qwen2 language-model matmuls (`attn_{q,k,v,o}`, `ffn_{gate,up,down}`, `lm.output`)
75
  are quantized. The MoonViT vision tower, the projector, all norms and biases, and the two
76
- host-read f32 tensors (`lm.tok_embd`, `vit.pos_emb`) stay **f32** so the parity-sensitive
77
  vision path is untouched. q8_0/q6_k are box-identical; lower bit-widths trade a little box
78
  precision for size.
79
 
 
15
  library_name: gguf
16
  ---
17
 
18
+ # locate-anything.cpp - GGUF
19
 
20
  GGUF builds of [`nvidia/LocateAnything-3B`](https://huggingface.co/nvidia/LocateAnything-3B)
21
+ for **[locate-anything.cpp](https://github.com/mudler/locate-anything.cpp)** - a C++/ggml
22
  inference engine for open-vocabulary detection / visual grounding, no Python at inference time.
23
 
24
  **Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team.**
25
 
26
  The detections are the same as the official PyTorch implementation (the engine is
27
+ parity-gated against it), and it runs faster - on CPU and GPU.
28
 
29
  ## Files
30
 
31
  | File | Bits (LM) | Size | Notes |
32
  | ---- | --------- | ---- | ----- |
33
  | `locate-anything-f16.gguf` | f16 | ~9.2 GB | LM matmuls in f16, everything else f32 |
34
+ | `locate-anything-q8_0.gguf` | q8_0 | ~6.3 GB | near-lossless; **box-identical** to f32 - recommended |
35
  | `locate-anything-q6_k.gguf` | q6_k | ~5.5 GB | box-identical to f32 |
36
  | `locate-anything-q5_k.gguf` | q5_k | ~5.1 GB | sub-pixel box drift |
37
  | `locate-anything-q4_k.gguf` | q4_k | ~4.7 GB | smallest; sub-pixel box drift |
 
65
 
66
  Run against the official model exactly as its model card documents (bf16), greedily, on one
67
  GB10 GPU. Precision-matched (our **f16** vs its bf16) ours is **~1.7×** faster; the
68
+ recommended **q8_0** build (box-identical) is **~1.9-2.1×**:
69
 
70
  ![GB10 GPU speedup vs official bf16](gpu_speedup.png)
71
 
 
73
 
74
  Only the Qwen2 language-model matmuls (`attn_{q,k,v,o}`, `ffn_{gate,up,down}`, `lm.output`)
75
  are quantized. The MoonViT vision tower, the projector, all norms and biases, and the two
76
+ host-read f32 tensors (`lm.tok_embd`, `vit.pos_emb`) stay **f32** - so the parity-sensitive
77
  vision path is untouched. q8_0/q6_k are box-identical; lower bit-widths trade a little box
78
  precision for size.
79