LibrePEg14-cls

Perception Encoder (PE) Core PE-Core-G/14-448 packaged for LibreYOLO.

PE Core is a dual-tower vision-language encoder: it maps images, text, and whole videos into one normalized embedding space, and classifies zero-shot from text prompts.

from libreyolo import LibreYOLO

# zero-shot classification
model = LibreYOLO("LibrePEg14-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
r = model.predict("warehouse.jpg")[0]
print(model.names[r.probs.top1], float(r.probs.top1conf))

# embeddings: image, text, and whole video in one space
embedder = LibreYOLO("LibrePEg14-cls.pt", task="embed")
embedder.predict("photo.jpg")      # (1, 1280) image row
embedder.embed_text(["a dog"])     # (1, 1280) text row
embedder.predict("clip.mp4")       # (1, 1280) row for the whole clip

The bare canonical filename is the download trigger - there is no from_pretrained.

Memory note. g14 has a 1.88B-parameter vision tower. Expect roughly 8 GB of RAM/VRAM in float32 just to hold the weights, plus activations. Prefer l14 or smaller unless you specifically need the gigantic variant.

Specification

Family / size	pe / g14
Tasks	classify (zero-shot), embed
Input resolution	448x448
Embedding dimension	1280
Text context length	72
Preprocessing	RGB, bilinear square resize, mean/std = (0.5, 0.5, 0.5)
Video pooling	mean of independently encoded frames, L2-normalized once
Training	not supported (inference-only; train() raises)
Exports	ONNX, TorchScript

Source

• Upstream repository: timm/PE-Core-bigG-14-448
• Upstream revision: 17aa0c25addfa14198fa2ff73d845a22d433432e
• Upstream license: Apache-2.0

This is a converted OpenCLIP-compatible PE Core checkpoint, not an unmodified official facebook/PE-Core-* package checkpoint.

Modifications

State-dict metadata wrap only (weights/convert_pe_weights.py): LibreYOLO checkpoint metadata is added and tensors are moved to CPU. Learned parameters are unchanged.

Parity against unmodified open_clip_torch==3.2.0 on float32 CPU is exact (max_abs_diff == 0.0) for image embeddings, text embeddings, zero-shot logits, and fixed-frame video embeddings.

License

Apache-2.0, inherited from the upstream weights. See LICENSE and NOTICE in this repository. You are responsible for complying with those terms.