Ref-YOLO-World

Model weights for Ref-YOLO-World, an efficient referring expression comprehension (REC) model for edge devices. See the GitHub repository for installation, training, and evaluation.

File	Description
ref_yolo_world_x_multidataset.pth	Released Ref-YOLO-World model.
ref_yolo_world_x_multidataset_fp16.mlpackage	Core ML version of the released model (FP16 compute). It takes the cached text tensors as inputs and runs only the per-frame detector.
ref_yolo_world_x_multidataset_text_encoder_fp16.mlpackage	Core ML version of the LoRA-merged CLIP text encoder (FP16 compute). It turns token ids into the text tensors that the Core ML detector takes.
tokenizer/	CLIP tokenizer files (vocab.json, merges.txt) for the Core ML text encoder.
refcocog_val_text_features.pt	Text-feature cache for the cached inference path on RefCOCOg val.
onnx/	ONNX versions of the detector and the LoRA-merged text encoder, used by the browser demo. Both accept a variable token length.

With the two Core ML models, the whole pipeline runs on-device: encode the referring expression once with the text encoder, then run the detector on each frame.

Download

hf download sakusaku3939/Ref-YOLO-World ref_yolo_world_x_multidataset.pth --local-dir weights
hf download sakusaku3939/Ref-YOLO-World \
  --include "ref_yolo_world_x_multidataset_fp16.mlpackage/*" \
  --include "ref_yolo_world_x_multidataset_text_encoder_fp16.mlpackage/*" \
  --include "tokenizer/*" \
  --local-dir weights
hf download sakusaku3939/Ref-YOLO-World refcocog_val_text_features.pt --local-dir results/text_cache

License

GPL-3.0, following YOLO-World.