LibrePEt16-cls

Perception Encoder (PE) Core PE-Core-T/16-384 packaged for LibreYOLO.

PE Core is a dual-tower vision-language encoder: it maps images, text, and whole videos into one normalized embedding space, and classifies zero-shot from text prompts.

from libreyolo import LibreYOLO

# zero-shot classification
model = LibreYOLO("LibrePEt16-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
r = model.predict("warehouse.jpg")[0]
print(model.names[r.probs.top1], float(r.probs.top1conf))

# embeddings: image, text, and whole video in one space
embedder = LibreYOLO("LibrePEt16-cls.pt", task="embed")
embedder.predict("photo.jpg")      # (1, 512) image row
embedder.embed_text(["a dog"])     # (1, 512) text row
embedder.predict("clip.mp4")       # (1, 512) row for the whole clip

The bare canonical filename is the download trigger - there is no from_pretrained.

Specification


Family / size	`pe` / `t16`
Tasks	`classify` (zero-shot), `embed`
Input resolution	384x384
Embedding dimension	512
Text context length	32
Preprocessing	RGB, bilinear square resize, mean/std = (0.5, 0.5, 0.5)
Video pooling	mean of independently encoded frames, L2-normalized once
Training	not supported (inference-only; `train()` raises)
Exports	ONNX, TorchScript

Source

Upstream repository: timm/PE-Core-T-16-384
Upstream revision: 7fe539ed578ac49a1c2b4f946e4b0747704c825a
Upstream license: Apache-2.0

This is a converted OpenCLIP-compatible PE Core checkpoint, not an unmodified official facebook/PE-Core-* package checkpoint.

Modifications

State-dict metadata wrap only (weights/convert_pe_weights.py): LibreYOLO checkpoint metadata is added and tensors are moved to CPU. Learned parameters are unchanged.

Parity against unmodified open_clip_torch==3.2.0 on float32 CPU is exact (max_abs_diff == 0.0) for image embeddings, text embeddings, zero-shot logits, and fixed-frame video embeddings.

License

Apache-2.0, inherited from the upstream weights. See LICENSE and NOTICE in this repository. You are responsible for complying with those terms.

Downloads last month: -; Downloads are not tracked for this model. How to track