See our collection for all versions of MetaCLIP 2.

Run MetaCLIP 2 with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/metaclip2_worldwide_m16_384

Paper: MetaCLIP 2: A Worldwide Scaling Recipe (arXiv:2507.22062) · HF Papers

MetaCLIP 2 is a CLIP-architecture dual encoder trained on a worldwide, multi-language data distribution (not an English-filtered crawl). Worldwide variants use a 901k-token XLM-RoBERTa vocabulary instead of CLIP's English BPE; mt5 variants use SentencePiece. Class prompts can be non-English without a separate multilingual checkpoint.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of facebook/metaclip-2-worldwide-m16-384 for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a zero-shot image-text checkpoint (MetaClip2ZeroShotClassify). The quick start below scores the same concept in English / French / Spanish / German against one distractor.

✨ Quick start (multilingual zero-shot)

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

import keras
from kerasformers.models.metaclip2 import (
    MetaClip2Processor,
    MetaClip2ZeroShotClassify,
)

processor = MetaClip2Processor.from_weights("kerasformers/metaclip2_worldwide_m16_384")
model = MetaClip2ZeroShotClassify.from_weights("kerasformers/metaclip2_worldwide_m16_384")

# Same concept in four languages + one English distractor.
# Unlike CLIP (English BPE) / SigLIP v1, MetaCLIP 2 is trained worldwide.
labels = [
    "a photo of teddy bears",  # English
    "une photo d'ours en peluche",  # French
    "una foto de ositos de peluche",  # Spanish
    "ein Foto von Teddybären",  # German
    "a photo of a truck",  # distractor
]
inputs = processor(text=labels, image_paths="your_image.jpg")
output = model(
    {
        "images": inputs["images"],
        "token_ids": inputs["token_ids"],
        "padding_mask": inputs["padding_mask"],
    }
)
probs = keras.ops.convert_to_numpy(
    keras.ops.softmax(output["image_logits"], axis=-1)
).squeeze()
for label, p in zip(labels, probs):
    print(f"{p:.6f}  {label}")

Load any MetaCLIP 2 variant the same way with from_weights("kerasformers/<variant>"):

Variant	Hub	Notes
`metaclip2_worldwide_s16_224`	`kerasformers/metaclip2_worldwide_s16_224`
`metaclip2_worldwide_s16_384`	`kerasformers/metaclip2_worldwide_s16_384`
`metaclip2_worldwide_m16_224`	`kerasformers/metaclip2_worldwide_m16_224`
`metaclip2_worldwide_m16_384`	`kerasformers/metaclip2_worldwide_m16_384`
`metaclip2_worldwide_b16_224`	`kerasformers/metaclip2_worldwide_b16_224`
`metaclip2_worldwide_b16_384`	`kerasformers/metaclip2_worldwide_b16_384`
`metaclip2_worldwide_b32_224`	`kerasformers/metaclip2_worldwide_b32_224`
`metaclip2_worldwide_b32_384`	`kerasformers/metaclip2_worldwide_b32_384`
`metaclip2_worldwide_l14_224`	`kerasformers/metaclip2_worldwide_l14_224`
`metaclip2_worldwide_huge_quickgelu`	`kerasformers/metaclip2_worldwide_huge_quickgelu`
`metaclip2_worldwide_huge_378`	`kerasformers/metaclip2_worldwide_huge_378`
`metaclip2_worldwide_giant_224`	`kerasformers/metaclip2_worldwide_giant_224`
`metaclip2_worldwide_giant_378`	`kerasformers/metaclip2_worldwide_giant_378`
`metaclip2_mt5_worldwide_s16_224`	`kerasformers/metaclip2_mt5_worldwide_s16_224`	mT5 tokenizer
`metaclip2_mt5_worldwide_m16_224`	`kerasformers/metaclip2_mt5_worldwide_m16_224`	mT5 tokenizer
`metaclip2_mt5_worldwide_b32_224`	`kerasformers/metaclip2_mt5_worldwide_b32_224`	mT5 tokenizer

Tips

Set KERAS_BACKEND before importing Keras / kerasformers.
Prefer Processor.from_weights(...) so image size and tokenizer match the variant.
Processor returns token_ids / padding_mask (not input_ids / attention_mask).
Prompts can be non-English; keep accents exact (e.g. Teddybären vs Teddybaren).
Some large variants may convert on the fly from upstream if no prebuilt kerasformers weights are present.
See MetaCLIP 2 docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. MetaClip2ZeroShotClassify.from_weights("hf:facebook/metaclip-2-worldwide-m16-384").