IFM/PAN2-guardrail-Q16

Q16 for CLIP ViT-L/14

The output guard of the PAN2 guardrail: the Q16 inappropriate-content classifier with the OpenAI CLIP ViT-L/14 image encoder. A frame's probability of being inappropriate is

softmax(100 * cos(clip_image(frame), [e_appropriate, e_inappropriate]))[1]

Files

• model.safetensors
  • visual.*: the OpenAI CLIP ViT-L/14 image encoder, unmodified, in float16 with float32 layer norms and embeddings (the dtypes OpenAI CLIP runs it in on GPU).
  • embeddings: the two Q16 prompt embeddings [appropriate, inappropriate], float16 [2, 768], unmodified.
• config.json: the arguments of clip.model.VisionTransformer.

Converted with scripts/convert_q16_to_safetensors.py from:

• the OpenAI CLIP ViT-L-14.pt checkpoint (SHA-256 b8cca3fd41ae0c99ba7e8951adf17d267cdb84cd88be6f7c2e0eca1737a03836);
• data/ViT-L-14/prompts.p of ml-research/Q16 at commit 33de788 (SHA-256 9b6c411824c2f64e0f847aa3c9f2b15b8e3a6c75e0efd82f2f672670c97ccad5).

Usage

from pan2_guardrail import PAN2SafetyChecker

checker = PAN2SafetyChecker.from_pretrained().to("cuda")
checker.check_video_safety(frames)  # uint8 [T, H, W, 3]; the frames when they pass, None when blocked

License and Attribution

This repository holds the output guardrail of PAN 2: CLIP's image encoder and Q16's pre-trained weights. MBZUAI's own files in this repository are released under the Apache License, Version 2.0. The CLIP and Q16 files and weights remain under their own licenses.

The license files are:

• LICENSE: the license for MBZUAI's own files in this repository.
• NOTICE: MBZUAI's copyright notice and attribution for the third-party components.
• THIRD_PARTY_LICENSES: the full license text of each third-party component, with its original copyright line.

Modifications

The CLIP files and weights are used as released by their authors and have not been modified or fine-tuned by MBZUAI. PAN 2 uses Q16 for content moderation. Q16's pre-trained weights (the learned prompt parameters applied to CLIP) are used as released by its authors, who trained them on the SMID dataset; no SMID images are included in this repository. MBZUAI has only converted the CLIP image encoder and the Q16 weights into a single safetensors file; CLIP's text encoder is not included.