README.md

---
license: other
license_name: nvidia-non-commercial
license_link: LICENSE
base_model: nvidia/LocateAnything-3B
base_model_relation: quantized
pipeline_tag: image-text-to-text
inference: false
quantized_by: shadowrock-io
library_name: vllm
metrics:
  - f1
model-index:
  - name: LocateAnything-3B-Community-FP8-Dynamic
    results:
      - task:
          type: object-detection
          name: Open-vocabulary visual grounding
        dataset:
          name: COCO val2017 grounding slice (100 images, 288 queries)
          type: coco
          config: default
          split: validation
        metrics:
          - type: f1_at_iou_0.5
            name: F1@0.5 (seed mean)
            value: 0.6192
          - type: mean_f1_50_95
            name: mean F1@0.50:0.95 (seed mean)
            value: 0.4712
          - type: precision_at_iou_0.5
            name: Precision@0.5 (seed mean)
            value: 0.7255
          - type: recall_at_iou_0.5
            name: Recall@0.5 (seed mean)
            value: 0.5402
          - type: mean_matched_iou
            name: Mean matched IoU (seed mean)
            value: 0.8548
        source:
          name: ShadowRock eval (raw JSON)
          url: https://huggingface.co/shadowrock-io/LocateAnything-3B-Community-FP8-Dynamic/tree/main/evidence
tags:
  - fp8
  - w8a8
  - compressed-tensors
  - llm-compressor
  - vllm
  - visual-grounding
  - object-detection
  - vision-language
  - open-vocabulary-detection
  - quantized
  - safetensors
  - 8-bit
  - coco
  - qwen2
---
<a href="https://shadowrock.io">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://a.shadowrock.team/assets/logos/full/shadowrock-logo-white.svg">
    <img alt="ShadowRock" src="https://a.shadowrock.team/assets/logos/full/shadowrock-logo-black.svg" width="340">
  </picture>
</a>

# LocateAnything-3B — Community FP8-Dynamic

**Unofficial community quantization — not an NVIDIA release.**

FP8 W8A8 build of [nvidia/LocateAnything-3B](https://huggingface.co/nvidia/LocateAnything-3B) (revision [`c32291ca`](https://huggingface.co/nvidia/LocateAnything-3B/tree/c32291ca5e996f5a7a485845b4f57a233936bba0)), validated against a BF16 baseline of the published checkpoint. All credit for the model belongs to NVIDIA; this repo changes only the numeric precision of the decoder weights. Checkpoint size drops from 7.6 GB to 4.0 GB, and on our COCO val2017 grounding benchmark the FP8 model scores inside the BF16 model's own seed-to-seed variance band on every metric.

Pick this variant for Ada/Hopper/Blackwell GPUs where you want near-lossless behavior with native FP8 execution. The companion [NVFP4 build](https://huggingface.co/shadowrock-io/LocateAnything-3B-Community-NVFP4) halves the footprint again for memory-constrained Blackwell deployments.

## What is quantized

Scheme: `FP8_DYNAMIC` via [llm-compressor](https://github.com/vllm-project/llm-compressor) — per-channel FP8 weights, dynamic per-token FP8 activations, data-free. Only the Qwen2.5-3B language-model decoder linears are quantized. Everything the grounding head depends on stays BF16:

- MoonViT vision tower and the `mlp1` connector
- `embed_tokens` / `lm_head` (tied), which protects the `<0>`–`<1000>` coordinate token rows

## Validation

All evaluation drives the model through its own hybrid parallel-box-decoding runtime (`batch_utils.generate_batch_hybrid`) under the model's canonical sampling settings (temperature 0.7, top_p 0.9, repetition_penalty 1.1) with the torch RNG reseeded per case, which makes runs bit-reproducible on a fixed stack.

**Grounding benchmark, 2×2 design.** 100 COCO val2017 images (first 100 ids with annotations, ascending — a deterministic slice), one query per ground-truth category per image, 288 queries. Two seeds per model, so the BF16 seed-to-seed spread is measured rather than assumed. LocateAnything emits boxes without confidence scores, so we report score-free greedy-matched precision/recall/F1 instead of score-ranked mAP; the paper's own headline metric family is mean F1.

| model / seed | P@0.5 | R@0.5 | F1@0.5 | mean F1 (0.50:0.95) | mean matched IoU |
|---|---|---|---|---|---|
| BF16, seed A | 0.7345 | 0.5501 | 0.6290 | 0.4761 | 0.8536 |
| BF16, seed B | 0.7329 | 0.5303 | 0.6154 | 0.4691 | 0.8559 |
| FP8, seed A | 0.7199 | 0.5472 | 0.6218 | 0.4750 | 0.8560 |
| FP8, seed B | 0.7311 | 0.5331 | 0.6166 | 0.4674 | 0.8535 |

The BF16 model moves 0.0136 F1@0.5 between seeds; FP8 sits inside that band on every metric, with same-seed deltas 0.1–0.5× the seed variance. The paper reports COCO mean F1 ≈ 52.1 for the full protocol; our BF16 baseline lands ≈ 47.6 under the deviations listed below, and the number that matters here is FP8-vs-BF16 under an identical protocol.

**Contract test.** The quantized export passes the same gate as the BF16 snapshot: parallel-box-decoding forwards active (`mtp_forwards > 0`), coordinate token ids intact and finite, tied lm_head verified, valid labeled boxes on the fixture set.

**Fixture parity.** On the 12-case fixture set, 10/12 responses are exact or near-exact (coordinate deltas within tolerance); the 2 remaining are crowded scenes where sampling legitimately picks a different box order.

**Teacher-forced diagnostics.** Full-sequence forced decoding over the BF16 golden responses shows all top-1 flips confined to coordinate positions (no label or structure flips), mostly ±1–4 coordinate units on a 0–1000 grid.

Raw results ship in this repo under [`evidence/`](evidence/): per-query benchmark predictions for both seeds, fixture parity report, and the teacher-forced capture. Reproduction scripts (quantizer, contract test, teacher-forced capture, COCO benchmark harness) live in the [source repo](https://git.srk.rest/shadowrock/locateanything-quant).

### Protocol deviations from the paper

- 100-image deterministic slice of val2017, not the full split
- greedy best-first IoU matching (score-free), not a ranked matcher
- queries are per-GT-category per image (assumes category presence is known)

## Usage

### vLLM (true W8A8)

vLLM reads the compressed-tensors config and runs real FP8 weight and activation kernels:

```python
from vllm import LLM
llm = LLM(model="shadowrock-io/LocateAnything-3B-Community-FP8-Dynamic", trust_remote_code=True)
```

### transformers (decompress-on-load)

`run_compressed: false` is set in `config.json`: transformers decompresses the FP8 weights to BF16 at load. Numerics of the quantized weights are preserved, but activations run in BF16, so this path measures weight-quantization error only. Pin `transformers==4.57.1` and `compressed-tensors==0.12.2`; the model's inference contract (sampling settings, hybrid decode runtime) is documented in the upstream README, which ships in this repo together with the pinned model code.

```python
from transformers import AutoModel
model = AutoModel.from_pretrained(
    "shadowrock-io/LocateAnything-3B-Community-FP8-Dynamic",
    trust_remote_code=True, torch_dtype="bfloat16")
```

Greedy decoding is not supported by the base model (it never emits the stop token); use the canonical sampling settings above.

## Prior art

Other community quantizations of this model exist (GGUF, MLX 4/8-bit, ONNX INT4, an NVFP4). None we found publish accuracy measurements against the BF16 baseline; this release's purpose is quantization with evidence. Reproduction scripts (quantizer, contract test, teacher-forced capture, COCO benchmark) live in the [source repo](https://git.srk.rest/shadowrock/locateanything-quant).

## Intended use & limitations

Intended uses are the base model's: open-vocabulary visual grounding and object detection from natural-language queries. The base card's intended-use, safety, and coverage statements — [nvidia/LocateAnything-3B](https://huggingface.co/nvidia/LocateAnything-3B) — carry over unchanged; quantization alters none of the model's behavior boundaries, only its numeric precision. Our evaluation establishes parity on the COCO benchmark above and nothing beyond it: other image domains, query styles, and hardware/runtime combinations inherit the base model's behavior with quantization noise that we have not measured there.

## Attribution & citation

Quantization, validation harness, and card by [Matt Busi](https://www.linkedin.com/in/matt-busi) ([@mattbusi](https://huggingface.co/mattbusi) on Hugging Face) at [ShadowRock](https://shadowrock.io). If you use this build, cite the NVIDIA base model — the grounding capability is theirs:

```bibtex
@misc{nvidia2026locateanything,
  title  = {LocateAnything-3B},
  author = {NVIDIA},
  year   = {2026},
  url    = {https://huggingface.co/nvidia/LocateAnything-3B}
}
```

## License

NVIDIA non-commercial license, inherited unchanged from the base model — see [LICENSE](LICENSE). Quantized weights are a derivative of NVIDIA's checkpoint and carry the same terms. No COCO images are redistributed here; the benchmark fetches them by id from cocodataset.org. Community build by [ShadowRock](https://shadowrock.io); no NVIDIA affiliation or endorsement.

## About ShadowRock

[ShadowRock](https://shadowrock.io) is an AI-specialized systems integrator and Zendesk Premier Partner. We help businesses get real value from their go-to-market technology, from CRM and support platforms to applied AI like the models in this collection. Find us at [shadowrock.io](https://shadowrock.io) or on [LinkedIn](https://www.linkedin.com/company/shadowrock).