README.md

---
language:
  - en
  - it
  - es
  - fr
  - de
license: apache-2.0
library_name: transformers
tags:
  - sentiment-analysis
  - text-classification
  - multilingual
  - restaurants
  - 5-star
base_model: jhu-clsp/mmBERT-base
pipeline_tag: text-classification
---

# 🍜 Multilingual Restaurant Review Sentiment Model 🌍

Hey there! This isn't just _another_ sentiment model. This is a fine-tuned powerhouse specifically designed to understand the nuance of 1-to-5 star restaurant reviews across **5 different languages**.

It was trained on a massive, perfectly balanced dataset of **400,000+ real, human-written, reviews** and achieves state-of-the-art performance.

## ✨ Model Features

- **Multilingual:** Trained on **English**, **Italian**, **Spanish**, **French**, **German**.
- **5-Star Specialist:** Predicts ratings on a 1-5 star scale.
- **SOTA Performance:** Achieves an incredibly low **MAE of ~0.29**. (More on that below!)

---

## 🎯 Just How Good Is It? (Performance)

Forget accuracy. For star ratings, **Mean Absolute Error (MAE)** is what matters. It measures how "off" the prediction is.

What does that mean? It means on average, the model's prediction is **only off by 0.29 stars**.

- It _knows_ a 5-star is close to a 4-star.
- It _knows_ a 1-star is NOT a 5-star.
- It **rarely** confuses a positive review for a negative one.

Here are the full results from the validation set (500k real-world reviews!):

| Metric       | Score     | Why it Matters                                               |
| :----------- | :-------- | :----------------------------------------------------------- |
| **MAE**      | **0.293** | 🏆 **The model main score.**                                 |
| **Accuracy** | 78.2%     | How often the model guess the _exact_ star (after rounding). |
| **Macro F1** | 0.683     | Shows it's good at all classes, not just the majority class. |
| **MSE**      | 0.182     | The loss the model was trained on (Mean Squared Error).      |

---

### Confusion Matrix

This shows where the model makes its errors. As you can see, almost all errors are "off-by-one" (like predicting a 4 for a 5-star), which is exactly what we want.

|            | **Predicted 1** | **Predicted 2** | **Predicted 3** | **Predicted 4** | **Predicted 5** |
| :--------- | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: |
| **True 1** |      14683      |      8391       |       568       |       44        |       34        |
| **True 2** |      2504       |      13699      |      4068       |       95        |       13        |
| **True 3** |       290       |      6271       |      23824      |      5700       |       229       |
| **True 4** |       18        |       267       |      6940       |      66361      |      25089      |
| **True 5** |       44        |       143       |       553       |      47873      |     272298      |

---

### Performance Per Language

The model performs strongly across all five languages. Here is the final accuracy for each language on the test set:

| Region    | Accuracy |
| :-------- | :------- |
| `English` | 0.827    |
| `Italian` | 0.778    |
| `Spanish` | 0.775    |
| `French`  | 0.763    |
| `German`  | 0.755    |

---

## 🧠 The "Regression Trick" (Why it's so good)

Most models do "classification" (is it A, B, or C?). This is a bad fit for star ratings.

This model was trained as a **regression** task. It predicts a single number (like 4.7, 1.2, or 3.5) instead of just "5-star". This teaches the model that 4-stars are "closer" to 5-stars than 1-star is, which is how it gets such a low MAE.

---

## 🚀 How to Use

Since this is a regression model, the output is a single float number. You'll want to round it to get a final "star" rating.

### ⚠️ A Critical Note on Input Format

**This is very important for getting the best performance!**

This model was not just trained on review text; it was trained using a specific format that includes **both the review title and the review text**, separated by the `[SEP]` token.

The title often contains a powerful summary of the sentiment (e.g., "Best Pasta Ever!" or "Total Rip-off!"). Using this format ensures the model gets the same type of input it was trained on.

**Correct Format:**
`input_text = review_title + " [SEP] " + review_text`

If you only have the review text, the model will still work well, but performance will be slightly lower.

### Pipeline Usage Example

Here is how you should format your inputs before passing them to the pipeline:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
import numpy as np # Make sure to import numpy

model_name = "Festooned/Multilingual-Restaurant-Reviews-Sentiment"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# ---
# IMPORTANT: This model predicts a single number (regression).
# ---

# Let's create a pipeline
sentiment_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)

# Example reviews using the recommended format
reviews = [
    "Absolutely incredible [SEP] This was the best pasta I've ever had in my life.", # 5-star
    "Servicio terrible [SEP] El servicio fue terrible y la comida tardó una hora en llegar.", # 1-star
    "It was fine [SEP] It was... fine. Nothing special, but not bad either." # 3-star
]

# Get the raw predictions
raw_preds = sentiment_pipe(reviews)
print(raw_preds)
# [{'label': 'LABEL_0', 'score': 4.81}]
# [{'label': 'LABEL_0', 'score': 1.12}]
# [{'label': 'LABEL_0', 'score': 2.95}]

# ---
# How to get the actual "star rating"
# (Remember our labels are 0-4, so we add 1)
# ---
for text, pred in zip(reviews, raw_preds):
    # 'score' is the raw regression value (our model predicts 0-4)
    raw_score = pred['score']

    # Round and clamp to be safe (0-4)
    star_label_rounded = np.clip(round(raw_score), 0, 4)

    # Add 1 to get the 1-5 star rating
    final_star_rating = int(star_label_rounded + 1)

    print(f"Review: {text[:40]}...")
    print(f"  Final Rating: {final_star_rating} stars\n")
```

---

## 💡 Bonus: Convert to 3 Classes (Bad/Neutral/Good)

This 5-star model is flexible! If you don't need 5 classes, you can easily group the results.

Here's a simple helper function to convert the 1-5 star rating into **Bad**, **Neutral**, or **Good**.

```python
def to_3_class(rating):
    """Converts a 1-5 star rating into a 3-class sentiment."""
    # The 'rating' is the rounded 1-5 star value
    if rating <= 2:
        return "😞 Bad"
    elif rating == 3:
        return "😐 Neutral"
    else: # 4 or 5 stars
        return "😄 Good"

# Example using the rounded rating from the code above:
# Let's say a review got a rounded rating of 1
rating_1 = 1
print(f"Rating {rating_1} is: {to_3_class(rating_1)}")

# Let's say a review got a rounded rating of 3
rating_3 = 3
print(f"Rating {rating_3} is: {to_3_class(rating_3)}")

# Let's say a review got a rounded rating of 5
rating_5 = 5
print(f"Rating {rating_5} is: {to_3_class(rating_5)}")

Output:
Rating 1 is: 😞 Bad
Rating 3 is: 😐 Neutral
Rating 5 is: 😄 Good
```

---

## 🧪 Bonus: A Test of Specialization (Domain Shift)

This model is a SOTA-level _restaurant_ critic. But what happens if it's asked to review a car mechanic or a hair salon?

To find out, the model was tested on the **`yelp_review_full`** dataset. This dataset is **not** just restaurants; it includes reviews for auto shops, plumbers, gyms, salons, and all other business types.

The results are exactly what would be expected from a highly trained specialist:

| Metric       | Score on Restaurant-Only Data | Score on `yelp_review_full` (All businesses) |
| :----------- | :---------------------------: | :------------------------------------------: |
| **MAE**      |          **0.2928**           |                    0.4648                    |
| **Accuracy** |           **78.2%**           |                    62.7%                     |

---

## Citation

If you use this model in your research or app, please give it a shout-out!

```bibtex
@misc{adobati-2025-multilingual-restaurant,
  author = {Simone Adobati},
  title = {A Multilingual 5-Class Restaurant Review Sentiment Model},
  year = {2025},
  publisher = {Hugging Face},
  journal = {Hugging Face Model Hub},
  howpublished = {\url{[https://huggingface.co/](https://huggingface.co/)[Festooned/Multilingual-Restaurant-Reviews-Sentiment]}}
}
```