| --- |
| title: undertow |
| emoji: 🌊 |
| colorFrom: blue |
| colorTo: gray |
| sdk: static |
| pinned: false |
| license: mit |
| short_description: Four sentiment corpora, every model tested on all four |
| --- |
| |
| # undertow |
|
|
| A sentiment classifier reports one number: accuracy on a test set drawn from the same |
| corpus it trained on. **undertow** is about the other twelve. |
|
|
| Four corpora — IMDB, SST-2, Yelp, Amazon — two classifiers, and every model evaluated on |
| every domain. A bag of words and a linear probe on frozen DistilBERT features are nearly |
| tied on the test set everyone reports (86.0% vs 87.1%). On the three domains nobody |
| reports, the probe wins by 3.5 points and carries 29% less domain-shift penalty. Picking |
| the method on in-domain accuracy gets it wrong on two of the four corpora. |
|
|
| This page runs the TF-IDF arm live in your browser: type a review and the four models, |
| identical in every way except which 6,000 reviews they read, each give you a verdict. |
| Nothing is sent anywhere and no answer is precomputed — the JavaScript is a tested port of |
| the Python original, held to it at 1e-9. |
|
|
| Corpora: [IMDB](https://ai.stanford.edu/~amaas/data/sentiment/) (Maas et al., 2011), |
| [SST-2](https://gluebenchmark.com/tasks) (Socher et al., 2013), Yelp and Amazon polarity |
| (Zhang et al., 2015). |
|
|
| Method, controls and the full write-up: <https://github.com/UsmarHaider/undertow> |
|
|
