Spaces:
Sleeping
Sleeping
| # EUMORA — Product Feature Specification | |
| **Version 1.0 | May 2026 | Status: In Development** | |
| --- | |
| ## 1. What EUMORA Does | |
| EUMORA is a **lyric-to-song recommendation engine**. A user provides a piece of text — song lyrics, a mood description, a sentence — and EUMORA: | |
| 1. **Classifies the emotional content** of the text using a fine-tuned DeBERTa-v3-Base transformer model | |
| 2. **Maps that emotion** to a set of Spotify audio feature targets (valence, energy, danceability, tempo, mode) | |
| 3. **Queries the Spotify Recommendations API** with those targets to find real songs matching the emotional fingerprint | |
| 4. **Returns a ranked list of track recommendations** with per-feature match justification | |
| --- | |
| ## 2. Spotify Audio Features — What We Can and Can't Map | |
| Spotify's `/v1/audio-features` and `/v1/recommendations` use the following fields. Our emotion classifier | |
| operates on text only, so mappability depends on whether emotion correlates with that feature: | |
| | Spotify Field | Type | Range | Can We Map? | Reasoning | | |
| |--------------------|--------------|----------------|-------------|-----------| | |
| | `valence` | float | 0.0 – 1.0 | ✅ Yes | Directly correlates with positive/negative emotion | | |
| | `energy` | float | 0.0 – 1.0 | ✅ Yes | High for joy/anger/surprise, low for sadness/neutral | | |
| | `danceability` | float | 0.0 – 1.0 | ✅ Yes | High for joy/love/surprise, low for sadness/fear | | |
| | `tempo` | float (BPM) | ~50 – 200 | ✅ Yes | Fast for joy/anger, slow for sadness/fear | | |
| | `mode` | int | 0 (minor) / 1 (major) | ✅ Yes | Joy/love → major; sadness/anger/fear → minor | | |
| | `loudness` | float (dB) | -60 – 0 | ⚠️ Partial | Anger/joy → louder; sadness/neutral → quieter | | |
| | `speechiness` | float | 0.0 – 1.0 | ⚠️ Partial | Sarcasm/anger skew slightly higher; not strongly correlated | | |
| | `acousticness` | float | 0.0 – 1.0 | ❌ No | Depends on instrumentation, not text emotion | | |
| | `instrumentalness` | float | 0.0 – 1.0 | ❌ No | Depends on vocal presence, not text content | | |
| | `liveness` | float | 0.0 – 1.0 | ❌ No | Live vs. studio recording — unrelated to emotion | | |
| | `key` | int | 0 – 11 (Pitch) | ❌ No | Musical key — no emotion mapping | | |
| | `time_signature` | int | 3 – 7 | ❌ No | Rhythmic metre — no emotion mapping | | |
| | `duration_ms` | int | ms | ❌ No | Song length — unrelated | | |
| **Summary**: We can confidently map 5 features (`valence`, `energy`, `danceability`, `tempo`, `mode`), | |
| partially use 2 more (`loudness`, `speechiness`), and skip the remaining 6. | |
| --- | |
| ## 3. Emotion → Spotify Feature Targets | |
| These are the numeric ranges passed as `target_*` parameters to `/v1/recommendations`. | |
| Where the classifier returns a **full probability distribution**, the top-2 emotions are | |
| **blended by their probabilities** (weighted average) for a more nuanced query. | |
| | Emotion | `target_valence` | `target_energy` | `target_danceability` | `target_tempo` (BPM) | `target_mode` | `seed_genres` | | |
| |-----------|-----------------|-----------------|----------------------|---------------------|---------------|--------------------------| | |
| | joy | 0.85 | 0.82 | 0.80 | 128 | 1 (major) | pop, dance | | |
| | love | 0.75 | 0.52 | 0.62 | 96 | 1 (major) | romance, r-n-b | | |
| | sadness | 0.18 | 0.28 | 0.32 | 72 | 0 (minor) | sad, indie, acoustic | | |
| | anger | 0.18 | 0.88 | 0.58 | 148 | 0 (minor) | metal, hard-rock | | |
| | fear | 0.22 | 0.52 | 0.38 | 88 | 0 (minor) | ambient, dark | | |
| | surprise | 0.62 | 0.80 | 0.70 | 138 | 1 (major) | pop, electronic | | |
| | neutral | 0.50 | 0.38 | 0.50 | 100 | 1 (major) | chill, study | | |
| | sarcasm | 0.45 | 0.60 | 0.55 | 110 | 0 (minor) | alternative, indie | | |
| ### Blended Query Example | |
| Input: *"I'm excited but kind of scared"* | |
| → Classifier: 52% surprise + 35% fear | |
| → Blended targets: | |
| - `target_valence` = (0.52 × 0.62) + (0.35 × 0.22) = 0.40 | |
| - `target_energy` = (0.52 × 0.80) + (0.35 × 0.52) = 0.60 | |
| - `target_tempo` = (0.52 × 138) + (0.35 × 88) = 103 BPM | |
| --- | |
| ## 4. Core System Components | |
| ### 4.1 Emotion Classifier (`src/predict.py`) | |
| - **Model**: DeBERTa-v3-Base fine-tuned on ~59k samples (GoEmotions + dair-ai/emotion) | |
| - **Output**: 8 emotion classes — `sadness`, `joy`, `love`, `anger`, `fear`, `surprise`, `neutral`, `sarcasm` | |
| - **Sarcasm handling**: Bayesian prior adjustment (`target_sarcasm_prior`) corrects training-domain bias | |
| - **Output format**: Primary emotion + full probability distribution + confidence score + music context | |
| - **Current accuracy**: ~65.6% weighted F1 on validation; 95%+ on unambiguous direct expressions | |
| ### 4.2 Emotion → Feature Mapper (`src/spotify.py` — to be built) | |
| - Converts classifier output to numeric Spotify `target_*` parameters | |
| - Implements probability-weighted blending across top-2 emotions | |
| - Expands confidence score into `min_*/max_*` constraint windows: | |
| - High confidence (>0.75) → tight window (±0.08 around target) | |
| - Medium confidence (0.45–0.75) → medium window (±0.15) | |
| - Low confidence (<0.45) → wide window (±0.25), broader genre seeds | |
| ### 4.3 Spotify Integration (`src/spotify.py` — to be built) | |
| **Endpoints used:** | |
| - `GET /v1/recommendations` — Core call with `target_valence`, `target_energy`, `target_danceability`, | |
| `target_tempo`, `target_mode`, `seed_genres`, `limit` | |
| - `GET /v1/audio-features/{id}` — Fetch actual features of returned tracks for match scoring | |
| - `GET /v1/search` — Optional seed track lookup by title/artist | |
| **Authentication**: OAuth 2.0 Client Credentials flow (no user login needed for recommendations) | |
| ### 4.4 Recommendation Ranker (`src/spotify.py` — to be built) | |
| After Spotify returns candidates, EUMORA re-ranks using: | |
| - **Feature distance score**: Euclidean distance across [`valence`, `energy`, `danceability`, `tempo_norm`] | |
| between the emotion target vector and each track's actual audio features | |
| - **Diversity filter**: Max 2 tracks per artist in top-10 | |
| - **Output**: Ordered list of tracks with per-track match score (0–100) | |
| --- | |
| ## 5. Data Flow (End-to-End) | |
| ``` | |
| User Input: "City lights blur as I'm driving through the night" | |
| ↓ | |
| [ EmotionPredictor — src/predict.py ] | |
| DeBERTa-v3-Base → 8-class softmax → sarcasm calibration | |
| ↓ | |
| { emotion: "fear", confidence: 0.54, probabilities: {fear: 0.54, anger: 0.23, ...} } | |
| ↓ | |
| [ Feature Mapper — src/spotify.py ] | |
| Blend top-2 emotions (fear × 0.54 + anger × 0.23, normalised) | |
| → { target_valence: 0.19, target_energy: 0.73, target_danceability: 0.46, | |
| target_tempo: 122, target_mode: 0, seed_genres: ["dark", "hard-rock"] } | |
| ↓ | |
| [ Spotify Recommendations API ] | |
| GET /v1/recommendations?target_valence=0.19&target_energy=0.73&limit=20&seed_genres=dark,hard-rock | |
| ↓ | |
| 20 raw candidate tracks (with audio_features fetched per track) | |
| ↓ | |
| [ Ranker — src/spotify.py ] | |
| Euclidean distance scoring + artist diversity filter | |
| ↓ | |
| Top 10 tracks with match scores + Spotify links | |
| ↓ | |
| User Output (CLI / future: UI) | |
| ``` | |
| --- | |
| ## 6. CLI Interface | |
| | Command | What it does | | |
| |---|---| | |
| | `python main.py predict "text"` | Emotion only (existing) | | |
| | `python main.py recommend "text"` | Full pipeline → Spotify tracks *(to be added)* | | |
| | `python main.py recommend "text" --limit 5` | Top 5 recommendations | | |
| | `python main.py recommend "text" --genre rock` | Override seed genre | | |
| | `python main.py recommend "text" --no-blend` | Use top-1 emotion only, no blending | | |
| --- | |
| ## 7. What It Does NOT Do (Phase 1 Scope) | |
| - Does not map `acousticness`, `instrumentalness`, `liveness`, `key`, `time_signature` — no text-to-audio model for these | |
| - Does not personalize based on user listening history (no user OAuth) | |
| - Does not analyze audio files or waveforms (text-only) | |
| - Does not stream or play music directly | |
| - Does not guarantee sarcasm accuracy — calibration mitigates it | |
| --- | |
| ## 8. Planned Extensions (Phase 2+) | |
| | Feature | Phase | Description | | |
| |---|---|---| | |
| | User preference learning | 2 | OAuth login; bias toward user's top genres/artists | | |
| | Audio file analysis | 3 | Upload a recording; extract features via librosa | | |
| | Acousticness/instrumentalness mapping | 3 | Infer from audio+lyrics combined model | | |
| | Multimodal fusion | 4 | Combine lyrics emotion + audio features | | |
| | Web UI | 5 | Browser interface with Spotify 30s clip previews | | |
| | Playlist generation | 5 | Multi-text input → coherent 10-track playlist | | |
| --- | |
| ## 9. New Dependencies | |
| | Library | Version | Role | | |
| |---|---|---| | |
| | `spotipy` | `>=2.23.0` | Spotify Web API client — handles OAuth + all endpoint calls | | |
| Add to `requirements.txt`: | |
| ``` | |
| spotipy>=2.23.0 | |
| ``` | |