senti
Live demo
Drag a point across a mood map and find songs that feel the way you do.
Problem
Music apps sort by genre and artist, but people usually look for songs by feeling. senti (formerly Moodify) places songs on a valence (feels bad to feels good) by arousal (calm to wired) map, cut into 16 emotions from Russell's circumplex model, so you can browse by mood directly. No login needed.
What I built
- A BERT regressor that reads a song's lyrics and predicts valence and arousal directly, trained on 2,044 MERGE songs.
- A live map of 5,411 Billboard year-end hits (1960-2025), scored over their whole lyric, plus 254 held-out test songs the model never saw in training. Songs play 30-second previews and link to Spotify.
- A listener rating mode: people confirm a song's spot or drag it to where it belongs, a third of the time blind to the model's guess. Rated songs move toward the listeners' median placement.
- Audio from a frozen music model (MERT) run on each song's 30-second preview and blended with the lyrics score. The 2,568 hits with a preview are placed by the blend, and the rest by lyrics alone.
- An earlier four-class mood classifier (happy, angry, sad, relaxed) that scores 88.7% on 141 held-out MoodyLyrics songs, kept as the baseline.
Approach
16 emotions, two axes
The map follows Russell's circumplex: valence on x, arousal on y, cut into the 16 emotion words of Feldman Barrett & Russell (1998). A song's emotion is the section its predicted point falls in, and points near the centre are labelled Mixed.
Baseline: class probabilities to coordinates
The first version placed each song at the probability-weighted average of its four quadrant corners. That is an expected quadrant position, not a continuous estimate. It bunched songs near the diagonals and left 6 of the 16 sections empty.
V = p(happy) + p(relaxed) - p(angry) - p(sad) A = p(happy) + p(angry) - p(sad) - p(relaxed)
Direct valence/arousal regression
The current model is bert-base-cased with a two-output tanh head, trained to predict valence and arousal in [-1, 1]. It fills 13 of the 16 sections.
Adding audio
Each 30-second clip is embedded by MERT-v1-95M, frozen, in 5-second slices. A ridge regression maps the embeddings to valence and arousal, with the layer chosen on validation songs (layer 6). A linear blend of the lyrics and audio scores, fitted on validation songs, gives the final position. On the live map, the audio comes from each song's Apple preview, scored on a cloud GPU (Modal) in one batch.
Retrieval
Song coordinates are precomputed offline. The map ranks songs by distance to the selected point, so the live site is static and needs no GPU.
How it fits together
- Lyrics (fetched locally)
- BERT valence/arousal regressor (offline)
- MERT audio model on 30-second previews (offline, GPU)
- Static song catalog
- Interactive mood map + Spotify links
Results
- Valence correlation
- 0.72
- Lyrics model, Pearson r on 256 held-out MERGE songs (R² 0.52, MAE 0.23). Four-class baseline: 0.46.
- Arousal correlation
- 0.61
- Lyrics model, Pearson r on the same songs (R² 0.36, MAE 0.18). Four-class baseline: 0.11.
- Arousal with audio
- 0.61 → 0.77
- Lyrics + audio vs lyrics alone, Pearson r on the 206 test songs with audio clips (valence 0.72 → 0.77). Paired bootstrap 95% intervals for the gain exclude zero on both axes.
- Arousal vs listeners
- 0.57 → 0.85
- Lyrics alone vs lyrics + audio, Pearson r against the median placement from 15 study listeners on 100 songs (3+ raters each). Valence: 0.63 → 0.65, a gain within noise.
Limitations
- MERGE's valence and arousal come from AllMusic mood tags mapped through word norms, not from listeners rating each song, so the target is noisy.
- Predictions are pulled toward the centre. The test songs leave Nervous and Tense empty; with the Billboard hits every section has songs.
- 2,843 of the 5,411 hits have no Apple preview, so they are still placed by lyrics alone.
- Audio helps arousal far more than valence: with listeners, the blend barely beats lyrics on valence.
- The listener check is small (15 raters, 100 songs), and one rater repeating a song agrees with themself at about r 0.7, so single ratings are noisy too.
- The model was starting to overfit by its fifth and final epoch.
What's next
- Train on a larger set (Deezer) once lyric fetching is set up.
- Find previews for the hits that lack one, so audio covers the whole map.
- Grow the rating study past 15 listeners and re-check the blend.
Credits
Valence/arousal labels from MERGE (Louro et al., 2024, Zenodo 13939205), licensed CC BY-NC 4.0. senti is non-commercial, and lyrics are never stored in the repo.