Jeevaaaa/image-to-song-recommender

Image → Song Recommender

Given a photo, this pipeline suggests songs that match its visual mood — similar to Instagram's "suggested audio" feature. Built entirely with free, pretrained components (no custom model training).

How it works

1. Mood detection (zero-shot): CLIP scores the input image against a fixed set of mood text prompts (e.g. "a happy, energetic, joyful photo", "a calm, peaceful, relaxing photo") and returns a confidence weight per mood.
2. Mood → audio-feature mapping: each mood is mapped by hand to a target point in Spotify's audio-feature space — valence, energy, danceability, acousticness, tempo. The top-3 detected moods are blended (weighted by CLIP's confidence) into one target vector.
3. Song matching: a NearestNeighbors search (scikit-learn) over a Spotify tracks dataset finds the songs whose audio features are closest to that target vector.

No part of this pipeline is trained — it's a zero-shot composition of an existing vision-language model with a hand-authored mood→feature mapping and a nearest-neighbor lookup.

Files

• pipeline.py — full pipeline: image in, ranked song recommendations out
• requirements.txt — dependencies

Usage

from pipeline import recommend_songs
from PIL import Image

image = Image.open("your_photo.jpg")
mood_breakdown, songs = recommend_songs(image, n_songs=5)

for mood, weight in mood_breakdown:
    print(f"{mood}: {weight:.2f}")

for song in songs:
    print(song["track_name"], "-", song["artist"])

Note: pipeline.py expects a spotify_dataset.csv file in the same directory (not included in this repo — see Dataset section below) with columns: track_name, artists, valence, energy, danceability, acousticness, tempo, and optionally track_genre.

Dataset

Song matching is powered by the Spotify Tracks Dataset (114k tracks, CC0 license). Not redistributed here — download separately and place as spotify_dataset.csv alongside pipeline.py.

Limitations

• Mood categories are a fixed, hand-authored list (12 moods) — images outside these categories will be forced into the closest available mood.
• The mood → audio-feature mapping is a manual design choice, not learned from data.
• CLIP was trained on general image-text pairs, not specifically on aesthetic/mood classification, so mood detection is approximate.

License

MIT (this pipeline code). CLIP itself is licensed separately by OpenAI/Hugging Face.