README.md

---
license: mit
library_name: transformers
tags:
- perceptual-similarity
- feature-extraction
- image-feature-extraction
- dreamsim
pipeline_tag: image-feature-extraction
---

# DreamSim (ensemble) — HuggingFace format

Unofficial HuggingFace-format port of the **DreamSim** perceptual-similarity metric
(ensemble variant). Loads via `trust_remote_code` with no `dreamsim` / `peft` runtime
dependency; weights are the LoRA-merged ensemble stored as `model.safetensors`.

> **Not an official DreamSim release.** DreamSim was created by Shobhita Sundaram,
> Netanel Tamir, Stephanie Fu, and Richard Zhang — see the original repository
> https://github.com/ssundaram21/dreamsim and paper *"DreamSim: Learning New
> Dimensions of Human Visual Similarity using Synthetic Data"* (NeurIPS 2023).
> This repo re-packages their released ensemble for `transformers`.

## What it is

The ensemble concatenates the features of three ViT-B/16 backbones — DINO (`cls`),
CLIP (`embedding`), OpenCLIP (`embedding`) — each LoRA-adapted, then mean/L2-normalizes
(`embed_size = 1792`). Perceptual distance between two images is `1 − cos`.

This port has been verified against the official `dreamsim` package on 1,000 real
images: **max |Δ embedding| < 3e-7 and max |Δ distance| < 2e-7** — identical up to
float32 rounding.

## Usage

```python
import torch
from PIL import Image
from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained("bigshanedogg/dreamsim", trust_remote_code=True).eval()
proc = AutoImageProcessor.from_pretrained("bigshanedogg/dreamsim", trust_remote_code=True)

img_a = proc(images=Image.open("a.png"), return_tensors="pt")["pixel_values"]
img_b = proc(images=Image.open("b.png"), return_tensors="pt")["pixel_values"]

emb = model(pixel_values=img_a).embeddings          # (1, 1792)
dist = model.compute_distance(img_a, img_b)          # 1 - cos (higher = more different)
```

The image processor resizes to 224×224 (BICUBIC) and scales to `[0, 1]`; the
per-backbone mean/std normalization is applied inside the model.

## Licensing

- Port + original DreamSim code/weights: **MIT** — see [`LICENSE`](LICENSE).
- Vendored ViT backbone code (adapted from DINO, Meta AI): **Apache-2.0** — full text in
  [`LICENSE.apache-2.0.txt`](LICENSE.apache-2.0.txt); attribution + modification statement
  in [`NOTICE`](NOTICE).
- Merged backbone weights derive from DINO (Apache-2.0), CLIP (MIT), and OpenCLIP
  (MIT code, LAION-based weights). Review the upstream backbone terms — and, for
  OpenCLIP, the LAION data provenance — before any commercial use.