| --- |
| library_name: tpips |
| base_model: Qwen/Qwen3-VL-Embedding-8B |
| pipeline_tag: image-feature-extraction |
| tags: |
| - text-conditioned |
| - perceptual-similarity |
| - image-similarity |
| - vision-language |
| license: other |
| --- |
| |
| # TPIPS — Embedding (late fusion) |
|
|
| Text-conditioned perceptual image similarity, built on |
| [Qwen/Qwen3-VL-Embedding-8B](https://huggingface.co/Qwen/Qwen3-VL-Embedding-8B). This repo holds the |
| **`embedding`** checkpoint (one of three TPIPS models, each in its own repo — see |
| the table at the bottom). Code and full docs: [https://github.com/adobe-research/TPIPS](https://github.com/adobe-research/TPIPS). |
|
|
| **Late fusion.** Each `(text, image)` is encoded independently into an L2-normalised embedding. The pairwise score is `cos(e_a, e_b)` (higher = more similar). |
| Odd-one-out probabilities are a `softmax` over the three "other-pair" scores |
| divided by the temperature; 2AFC compares the two reference-candidate scores. |
| The pairwise score is the model's raw output (temperature is applied at the |
| probability step). |
|
|
| | Property | Value | |
| |----------|-------| |
| | Base model | Qwen/Qwen3-VL-Embedding-8B | |
| | Pairwise score | `cos(e_a, e_b)` | |
| | Fine-tuning | LoRA (r=16, α=32) on the LLM layers | |
| | Pooling | last-token | |
| | Temperature | 0.05 (applied at the probability step) | |
| | Prompt `X` | `Represent the similarity of the image based on X.` | |
|
|
| ## Usage |
| TPIPS supports Python 3.10 and later. Install matching PyTorch and torchvision builds from the [official PyTorch installer](https://pytorch.org/get-started/locally/), then install TPIPS: |
|
|
| ```bash |
| pip install tpips |
| ``` |
|
|
| Start with the recommended `embedding` model: |
|
|
| ```python |
| import tpips |
| from PIL import Image |
| |
| model = tpips.load_model("embedding", device="cuda") |
| a = Image.open("a.jpg").convert("RGB") |
| b = Image.open("b.jpg").convert("RGB") |
| |
| similarity = model.similarity(a, b, factor="lighting") # higher is more similar |
| distance = model.distance(a, b, factor="lighting") # lower is more similar |
| ``` |
|
|
| The first call downloads the selected TPIPS checkpoint and its Qwen backbone. A CUDA GPU is recommended; FlashAttention is optional. |
|
|
|
|
| ## The TPIPS models |
|
|
| | Model | Repo | |
| |-------|------| |
| | Embedding (late fusion) | [`sywang/TPIPS-Embed-Qwen3VL-8B`](https://huggingface.co/sywang/TPIPS-Embed-Qwen3VL-8B) | |
| | Early Fusion | [`sywang/TPIPS-EarlyFusion-Qwen3VL-8B`](https://huggingface.co/sywang/TPIPS-EarlyFusion-Qwen3VL-8B) | |
| | Activation Distance | [`sywang/TPIPS-ActDiff-Qwen3VL-8B`](https://huggingface.co/sywang/TPIPS-ActDiff-Qwen3VL-8B) | |
|
|
| ## License |
|
|
| TPIPS is provided under the [Adobe Research License](https://github.com/adobe-research/TPIPS/blob/main/LICENSE) for noncommercial research use. See the license for the complete terms. |
