Upload 4 files

README.md

@@ -1,14 +1,122 @@
----
-title: Multilingual Sentiment Analysis
-emoji: 🏢
-colorFrom: yellow
-colorTo: purple
-sdk: gradio
-sdk_version: 6.21.0
-python_version: '3.12'
-app_file: app.py
-pinned: false
-short_description: Analyze sentiment using a fine-tuned XLM-RoBERTa model.
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+
+# Multilingual Sentiment Analysis
+
+An end-to-end NLP project for three-class sentiment classification: **negative**, **neutral**, and **positive**. The repository contains two XLM-RoBERTa experiments:
+
+1. an archived English airline-tweet model, and
+2. the current general multilingual Twitter-sentiment model.
+
+The current model is exposed through a Gradio app and is evaluated both overall and separately for every supported language.
+
+## 🎮 Demo: Gradio App
+
+Run the interactive demo locally:
+```bash
+python app.py
+```
+
+![Gradio app batch-analysis demo](assets/demo-screenshot.png)
+
+## Results at a glance
+
+| Experiment | Data | Evaluation | Result |
+| --- | --- | --- | --- |
+| Airline tweets (legacy) | English airline tweets | Validation set | 84.24% accuracy, 0.842 weighted F1 |
+| General multilingual (current) | 8-language Twitter sentiment dataset | Held-out test set (6,960 tweets) | 66.52% accuracy, 0.663 macro F1 |
+
+The two scores are **not directly comparable**: the airline model was evaluated on a narrow, English-only domain, whereas the multilingual model is evaluated across eight languages and broader Twitter content.
+
+### Current multilingual model: confusion matrix
+
+<img src="results/multilingual/eval/confusion_matrix.png" width="500">
+
+### Airline-tweet model: evaluation plots
+
+<img src="results/airline_tweets/eval/confusion_matrix.png" width="400">
+
+<img src="results/airline_tweets/eval/loss_curve.png" width="400">
+
+
+### Current multilingual model: test macro F1 by language
+
+| Language | Macro F1 |
+| --- | ---: |
+| German | 0.731 |
+| French | 0.721 |
+| Portuguese | 0.699 |
+| English | 0.692 |
+| Spanish | 0.667 |
+| Arabic | 0.647 |
+| Italian | 0.625 |
+| Hindi | 0.516 |
+
+
+The neutral class is the most difficult overall (0.588 F1). Hindi is the weakest supported language, so this project does not claim equal quality across all languages.
+
+## How it works
+
+1. `multilingual_sentiment_analysis.preprocess` downloads and prepares the multilingual dataset.
+2. Text is cleaned by removing URLs and mentions, normalizing whitespace, and removing only the `#` marker. Unicode text, accents, emojis, and non-Latin scripts are preserved.
+3. `multilingual_sentiment_analysis.train` fine-tunes `xlm-roberta-base` for three-class sequence classification.
+4. `multilingual_sentiment_analysis.evaluate` reports overall metrics, per-language metrics, and a confusion matrix.
+5. `app.py` provides single-text and batch inference with the saved model.
+
+## Dataset
+
+The current experiment uses [Cardiff NLP's Tweet Sentiment Multilingual dataset](https://huggingface.co/datasets/cardiffnlp/tweet_sentiment_multilingual). It provides the same label mapping used by this project:
+
+```text
+0 = negative
+1 = neutral
+2 = positive
+```
+
+It includes Arabic, English, French, German, Hindi, Italian, Portuguese, and Spanish. The preparation script loads each language configuration separately so it can preserve the official train/validation/test split and retain the language for evaluation.
+
+## Repository layout
+
+```text
+src/
+  multilingual_sentiment_analysis/
+    config.py       # shared paths, labels, and dataset settings
+    preprocess.py   # dataset download, cleaning, tokenization, and saving
+    train.py        # fine-tuning entry point
+    evaluate.py     # test and per-language evaluation
+    infer.py        # lazy model loading and inference helpers
+data/
+  airline_tweets/ # archived English-only source data and prepared splits
+  multilingual/   # prepared multilingual splits
+results/
+  airline_tweets/ # archived English-only checkpoints
+  multilingual/   # current checkpoints and evaluation outputs
+sentiment_model/
+  airline-tweets-sentiment-model/         # archived English-only model
+  general-multilingual-sentiment-model/   # current app model
+```
+
+Data, checkpoints, and model weights are intentionally ignored by Git.
+
+
+## Training configuration
+
+- Base checkpoint: `xlm-roberta-base`
+- Maximum token length: 128
+- Epochs: 3
+- Learning rate: 2e-5 with the Trainer's linear schedule
+- Train/evaluation batch sizes: 16 / 32
+- Best checkpoint criterion: weighted F1 on the validation set
+- Random seed: 42
+
+The multilingual training run completed in approximately 4 minutes 40 seconds on an NVIDIA RTX 5060 Laptop GPU.
+
+## Limitations and next steps
+
+- The model was evaluated on Twitter-style text only; it is not validated for reviews, support tickets, or other domains.
+- It is evaluated for eight languages, not all languages supported by the XLM-R tokenizer.
+- Hindi and neutral-sentiment performance require further improvement.
+- A useful next experiment is to initialize from a Twitter-adapted multilingual encoder such as XLM-T and compare per-language macro F1 against this baseline.
+
+## Attribution
+
+The multilingual dataset is provided by Cardiff NLP. See its [dataset card](https://huggingface.co/datasets/cardiffnlp/tweet_sentiment_multilingual) and the associated [XLM-T paper](https://aclanthology.org/2022.lrec-1.27/).

app.py

@@ -0,0 +1,74 @@
+import gradio as gr
+
+from multilingual_sentiment_analysis.infer import predict, predict_batch
+
+
+def analyze_single(text: str):
+    if not text or not text.strip():
+        return "⚠️ Please enter some text", ""
+    try:
+        result = predict(text.strip())
+        emoji = {"positive": "🟢", "neutral": "🟡", "negative": "🔴"}.get(result["label"], "⚪")
+        return f"{emoji} {result['label'].upper()}", f"{result['confidence'] * 100:.1f}%"
+    except Exception as error:
+        return f"❌ Error: {error}", ""
+
+
+def analyze_batch(batch_text: str):
+    if not batch_text or not batch_text.strip():
+        return "⚠️ Please enter texts (one per line)", []
+    texts = [line.strip() for line in batch_text.splitlines() if line.strip()]
+    if not texts:
+        return "⚠️ Please enter at least one text", []
+    try:
+        results = predict_batch(texts)
+        emojis = {"positive": "🟢", "neutral": "🟡", "negative": "🔴"}
+        rows = [
+            [text[:70] + "..." if len(text) > 70 else text,
+             f"{emojis.get(result['label'], '⚪')} {result['label'].upper()}",
+             f"{result['confidence'] * 100:.1f}%"]
+            for text, result in zip(texts, results)
+        ]
+        sentiments = [result["label"] for result in results]
+        summary = (
+            f"✅ Analyzed {len(texts)} texts | 😊 Positive: {sentiments.count('positive')} | "
+            f"😐 Neutral: {sentiments.count('neutral')} | 😞 Negative: {sentiments.count('negative')}"
+        )
+        return summary, rows
+    except Exception as error:
+        return f"❌ Error: {error}", []
+
+
+custom_theme = gr.themes.Base(primary_hue="cyan", secondary_hue="slate").set(
+    body_background_fill="#000000", body_text_color="#00FFFF",
+    button_primary_background_fill="#00FFFF", button_primary_text_color="#000000",
+    button_primary_background_fill_hover="#00DDDD", block_title_text_color="#00FFFF",
+    block_label_text_color="#00FFFF", input_background_fill="#111111",
+    input_border_color="#00FFFF", input_placeholder_color="#666666", border_color_primary="#00FFFF",
+)
+
+with gr.Blocks(title="🌍 Multilingual Sentiment Analysis", theme=custom_theme) as demo:
+    gr.Markdown("# 🌍 Multilingual Sentiment Analysis")
+    gr.Markdown("Analyze sentiment using a fine-tuned XLM-RoBERTa model.")
+    with gr.Tabs():
+        with gr.TabItem("📝 Single Text"):
+            with gr.Row():
+                with gr.Column(scale=3):
+                    text_input = gr.Textbox(label="Enter text to analyze", placeholder="Type something to analyze...", lines=4)
+                with gr.Column(scale=1):
+                    analyze_btn = gr.Button("🔍 Analyze", size="lg", variant="primary")
+            with gr.Row():
+                sentiment_output = gr.Textbox(label="Sentiment", interactive=False)
+                confidence_output = gr.Textbox(label="Confidence", interactive=False)
+            analyze_btn.click(analyze_single, inputs=text_input, outputs=[sentiment_output, confidence_output])
+        with gr.TabItem("📚 Batch Analysis"):
+            batch_input = gr.Textbox(label="Enter multiple texts (one per line)", placeholder="Text 1...\nText 2...", lines=8)
+            batch_btn = gr.Button("🚀 Batch Analyze", size="lg", variant="primary")
+            batch_summary = gr.Textbox(label="Summary", interactive=False)
+            batch_results = gr.Dataframe(headers=["Text", "Sentiment", "Confidence"], label="Results", interactive=False)
+            batch_btn.click(analyze_batch, inputs=batch_input, outputs=[batch_summary, batch_results])
+    gr.Markdown("---\nBuilt with ❤️ using Gradio • XLM-RoBERTa")
+
+
+if __name__ == "__main__":
+    demo.launch()

requirements.txt

@@ -0,0 +1,10 @@
+transformers>=4.46,<6
+datasets>=2.19,<4
+scikit-learn>=1.4,<2
+matplotlib>=3.8,<4
+seaborn>=0.13,<1
+pandas>=2.2,<3
+torch>=2.2,<3
+accelerate>=0.30,<2
+gradio>=4.0,<7
+-e .

setup.py

@@ -0,0 +1,14 @@
+from setuptools import setup, find_packages
+
+setup(
+    name="multilingual_sentiment_analysis",
+    version="0.1.0",
+    packages=find_packages("src"),
+    package_dir={"": "src"},
+    install_requires=[
+        "transformers",
+        "datasets",
+        "torch",
+        "gradio"
+    ],
+)