LibrePEs16-cls

Perception Encoder (PE) Core PE-Core-S/16-384 packaged for LibreYOLO.

PE Core is a dual-tower vision-language encoder: it maps images, text, and whole videos into one normalized embedding space, and classifies zero-shot from text prompts.

from libreyolo import LibreYOLO

# zero-shot classification
model = LibreYOLO("LibrePEs16-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
r = model.predict("warehouse.jpg")[0]
print(model.names[r.probs.top1], float(r.probs.top1conf))

# embeddings: image, text, and whole video in one space
embedder = LibreYOLO("LibrePEs16-cls.pt", task="embed")
embedder.predict("photo.jpg")      # (1, 512) image row
embedder.embed_text(["a dog"])     # (1, 512) text row
embedder.predict("clip.mp4")       # (1, 512) row for the whole clip

The bare canonical filename is the download trigger - there is no from_pretrained.

Specification

Family / size pe / s16
Tasks classify (zero-shot), embed
Input resolution 384x384
Embedding dimension 512
Text context length 32
Preprocessing RGB, bilinear square resize, mean/std = (0.5, 0.5, 0.5)
Video pooling mean of independently encoded frames, L2-normalized once
Training not supported (inference-only; train() raises)
Exports ONNX, TorchScript

Source

  • Upstream repository: timm/PE-Core-S-16-384
  • Upstream revision: 3249a38eb1c432ec19231c6fe1774acb6a4e4efe
  • Upstream license: Apache-2.0

This is a converted OpenCLIP-compatible PE Core checkpoint, not an unmodified official facebook/PE-Core-* package checkpoint.

Modifications

State-dict metadata wrap only (weights/convert_pe_weights.py): LibreYOLO checkpoint metadata is added and tensors are moved to CPU. Learned parameters are unchanged.

Parity against unmodified open_clip_torch==3.2.0 on float32 CPU is exact (max_abs_diff == 0.0) for image embeddings, text embeddings, zero-shot logits, and fixed-frame video embeddings.

License

Apache-2.0, inherited from the upstream weights. See LICENSE and NOTICE in this repository. You are responsible for complying with those terms.

Downloads last month
80
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support