Locus-2B

Locus-2B grounds a referring expression in an image and returns its bounding box. It is Qwen/Qwen3-VL-2B-Instruct with a small box head, trained with the Locus (Locate Once, Check Until Settled) recipe:

1. The image is prefilled once; every expression about it runs as a short branch over the cached prefix.
2. The box head drafts the box in one pass.
3. The draft is rendered as the coordinate tokens the model would write, and the model verifies all of them in parallel by Jacobi iteration until the draft stops changing.

With exact acceptance the returned box is the model's own greedy coordinate-token answer, reached in a few passes instead of one decoding step per token.

Files

This repository holds only the weights training changed: the box head and the fully fine-tuned language model and vision merger (model.safetensors), plus locus_config.json. The vision encoder, tokenizer and remaining weights come from Qwen/Qwen3-VL-2B-Instruct, which from_pretrained downloads separately.

Usage

from PIL import Image
from locus.hub import from_pretrained, ground   # pip install -e <code repo>

m = from_pretrained("<org>/Locus-2B")
for r in ground(m, Image.open("street.jpg"), ["the man in the red shirt", "the bus on the left"]):
    print(r["expression"], r["box"], r["passes"])   # box = [x0, y0, x1, y1] in pixels

accept=1.0 (default) returns the greedy coordinate-token box; accept=0.7 or 0.5 uses typical acceptance (fewer passes, no longer exactly greedy); verify=False returns the head's one-pass draft. A CUDA GPU is required.

Results

Mean over the 8 RefCOCO / RefCOCO+ / RefCOCOg val and test splits, at 640² pixels. Passes: mean sequential passes per expression.

Output	Acc@0.5	Acc@0.75	Acc@0.9	Passes
head (one pass)	90.01	79.79	54.87	1
verified, exact (greedy)	89.30	80.31	59.13	5.60
verified, typical (α = 0.5)	89.60	80.49	58.95	3.43

For reference, on the same rows: Jag-0.8B (regression head) 88.16 / 76.51 / 48.76; LocateAnything-3B (box tokens) 88.68 / 80.82 / 65.55. Latency on one RTX A6000, batch 1, 512² inputs, CUDA graphs: head 43.5 ms, exact verification 102.2 ms per expression.

Training

Qwen3-VL-2B-Instruct with full fine-tuning of the language model and vision merger at lr 1e-5, head and verifier trained jointly, 36k steps = 2.00 epochs (Jag's budget). Training data: RefCOCO, RefCOCO+ and RefCOCOg train splits, with every image that appears in any evaluation split removed.

Limitations

• Trained on the RefCOCO family only. Small objects suffer: on Ref-L4's Objects365 images the model falls far below the zero-shot base model.
• Verification is slower than the head alone, and slower than a pure regression head.
• Exactness holds for this model's greedy decode; in bf16 it holds up to the rounding that also makes two greedy runs with different batch shapes differ.
• The model answers grounding queries only. Other questions should go to the base model.

License and data

Weights: Apache-2.0, derived from Qwen/Qwen3-VL-2B-Instruct (Apache-2.0). Training annotations are RefCOCO/+/g over COCO images; check those datasets' terms for your use.