LibreSigLIP2so400m-cls
SigLIP 2 shape-optimized 400M patch-14 (384 px) dual-tower zero-shot image
classifier, repackaged for LibreYOLO. The accuracy tier of the LibreSigLIP2
family. Open vocabulary: define your own class set at runtime with
set_classes, no training needed. Multilingual (SentencePiece, Gemma
vocabulary), with an optional multi_label=True sigmoid scoring mode.
Usage
from libreyolo import LibreYOLO
model = LibreYOLO("LibreSigLIP2so400m-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
r = model.predict("frame.jpg")[0]
print(model.names[r.probs.top1], float(r.probs.top1conf))
Requires the libreyolo[siglip2] extra (SentencePiece tokenizer). Note:
this model runs at 384 px with a 27-layer 1152-wide ViT; it is heavy on CPU,
prefer a GPU.
Source
Derived from google/siglip2-so400m-patch14-384
at commit e8e487298228002f3d8a82e0cd5c8ea9c567f57f.
Copyright (c) Google LLC. Licensed under the Apache License 2.0.
Reference implementations: google-research/big_vision
and the Hugging Face transformers SigLIP code (both Apache-2.0). This
fixed-resolution SigLIP 2 checkpoint uses the original SigLIP transformer
architecture (model_type: "siglip"); the NaFlex variants are not included.
Modifications
State-dict metadata wrap only (LibreYOLO v1.0 checkpoint schema). Learned
parameters are unchanged. See weights/convert_siglip2_weights.py in the
LibreYOLO source repository.
Parity vs the reference transformers model at fp32 CPU: text tower and
vision encoder are bit-identical (max_abs_diff == 0); the attention-pooled
image feature differs by < 1e-6 (a torch MultiheadAttention fused-kernel
residual, documented in weights/parity_siglip2.py).
Zero-shot accuracy
Imagenette (10-class, val, model.val() zero-shot, prompt "a photo of a {}."):
99.5 top-1 (160 px source images), 99.6 top-1 (320 px source images). Beats
LibreSigLIP2b16-cls on both.
License
Apache License 2.0. See the LICENSE and NOTICE
files in this repository. Weights were trained by Google on the WebLI
dataset, which is not redistributed here.