havaferber/dermaglow-embeddings

DermaGlow Embeddings — Recommendation Engine (Part 3)

Precomputed image embeddings for the DermaGlow dataset, powering the "people with skin like yours" recommender in the DermaGlow app.

Model selection — 3 candidates evaluated

Metric: do the top-5 visually-nearest faces share the same skin labels? (random baseline: 19.6%)

Model	Mean label match	Dim	Embed time (1,100 imgs)
openai/clip-vit-base-patch32 (winner)	43.5%	512	27s
google/siglip-base-patch16-224	41.5%	768	82s
facebook/dinov2-small	36.2%	384	32s

On the final dataset, CLIP top-5 matches: skin_tone 61.2%, age_group 65.3%, skin_type 27.5%, main_concern 19.7%.

Design insight: visual embeddings capture tone/age strongly but skin concerns only weakly (they're subtle in photos). The app therefore uses a hybrid matcher: CLIP visual similarity + boosting by the user's questionnaire answers.

Files

• clip_embeddings.parquet — id + 512-dim L2-normalized vector per dataset row
• eval_results.json — full evaluation numbers

Usage

import pandas as pd, numpy as np
from huggingface_hub import hf_hub_download
df = pd.read_parquet(hf_hub_download("havaferber/dermaglow-embeddings", "clip_embeddings.parquet"))
emb = np.array(df["embedding"].tolist(), dtype=np.float32)   # (1100, 512), pre-normalized
# similarity = emb @ query_vec  (cosine)

Embedding model to encode new queries: openai/clip-vit-base-patch32 (get_image_features).