title: undertow
emoji: 🌊
colorFrom: blue
colorTo: gray
sdk: static
pinned: false
license: mit
short_description: Four sentiment corpora, every model tested on all four
undertow
A sentiment classifier reports one number: accuracy on a test set drawn from the same corpus it trained on. undertow is about the other twelve.
Four corpora — IMDB, SST-2, Yelp, Amazon — two classifiers, and every model evaluated on every domain. A bag of words and a linear probe on frozen DistilBERT features are nearly tied on the test set everyone reports (86.0% vs 87.1%). On the three domains nobody reports, the probe wins by 3.5 points and carries 29% less domain-shift penalty. Picking the method on in-domain accuracy gets it wrong on two of the four corpora.
This page runs the TF-IDF arm live in your browser: type a review and the four models, identical in every way except which 6,000 reviews they read, each give you a verdict. Nothing is sent anywhere and no answer is precomputed — the JavaScript is a tested port of the Python original, held to it at 1e-9.
Corpora: IMDB (Maas et al., 2011), SST-2 (Socher et al., 2013), Yelp and Amazon polarity (Zhang et al., 2015).
Method, controls and the full write-up: https://github.com/UsmarHaider/undertow