README.md

metadata

library_name: tpips
base_model: Qwen/Qwen3-VL-Embedding-8B
pipeline_tag: image-feature-extraction
tags:
  - text-conditioned
  - perceptual-similarity
  - image-similarity
  - vision-language
license: other

TPIPS — Embedding (late fusion)

Text-conditioned perceptual image similarity, built on Qwen/Qwen3-VL-Embedding-8B. This repo holds the embedding checkpoint (one of three TPIPS models, each in its own repo — see the table at the bottom). Code and full docs: https://github.com/adobe-research/TPIPS.

Late fusion. Each (text, image) is encoded independently into an L2-normalised embedding. The pairwise score is cos(e_a, e_b) (higher = more similar). Odd-one-out probabilities are a softmax over the three "other-pair" scores divided by the temperature; 2AFC compares the two reference-candidate scores. The pairwise score is the model's raw output (temperature is applied at the probability step).

Property	Value
Base model	Qwen/Qwen3-VL-Embedding-8B
Pairwise score	cos(e_a, e_b)
Fine-tuning	LoRA (r=16, α=32) on the LLM layers
Pooling	last-token
Temperature	0.05 (applied at the probability step)
Prompt X	Represent the similarity of the image based on X.

Usage

TPIPS supports Python 3.10 and later. Install matching PyTorch and torchvision builds from the official PyTorch installer, then install TPIPS:

pip install tpips

Start with the recommended embedding model:

import tpips
from PIL import Image

model = tpips.load_model("embedding", device="cuda")
a = Image.open("a.jpg").convert("RGB")
b = Image.open("b.jpg").convert("RGB")

similarity = model.similarity(a, b, factor="lighting")  # higher is more similar
distance = model.distance(a, b, factor="lighting")      # lower is more similar

The first call downloads the selected TPIPS checkpoint and its Qwen backbone. A CUDA GPU is recommended; FlashAttention is optional.

The TPIPS models

Model	Repo
Embedding (late fusion)	sywang/TPIPS-Embed-Qwen3VL-8B
Early Fusion	sywang/TPIPS-EarlyFusion-Qwen3VL-8B
Activation Distance	sywang/TPIPS-ActDiff-Qwen3VL-8B

License

TPIPS is provided under the Adobe Research License for noncommercial research use. See the license for the complete terms.