rishigupta04/yt-comments-sentiment-analyzer

🎬 YouTube Comment Sentiment Analyzer

Fine-Tuned Twitter-RoBERTa for Real-World YouTube Comments

---

🎯 Real-World Performance

Metric	Score
Accuracy	88.00%
Macro F1	87.68%
Weighted F1	87.73%

Evaluated on a custom real-world YouTube benchmark containing slang, emojis, internet culture references, and creator-style comments.

---

🚀 Overview

This model is a sentiment classifier specifically adapted for YouTube comments.

The backbone model, Twitter-RoBERTa, was originally trained on Twitter/X posts and social media text.

To make it understand YouTube comment culture, the model was further fine-tuned on a dataset containing 1 Million+ labeled YouTube comments.

The result is a model that better understands:

✅ Emojis

✅ Internet slang

✅ Creator-audience interactions

✅ Short-form comments

✅ Social media language

✅ YouTube culture

---

🧠 Model Architecture

Twitter/X Posts
        ↓
Twitter-RoBERTa
(cardiffnlp/twitter-roberta-base-sentiment-latest)
        ↓
Fine-Tuning on 1M+ YouTube Comments
        ↓
YouTube Comment Sentiment Analyzer
        ↓
Negative | Neutral | Positive

---

📊 Project Pipeline

---

🎯 Sentiment Classes

Label	Meaning
🔴 Negative	Criticism, dislike, frustration, complaints
⚪ Neutral	Informational, factual, objective comments
🟢 Positive	Praise, appreciation, excitement, support

---

🧪 Real-World Benchmark Results

A manually curated benchmark was created to simulate actual YouTube comments.

The benchmark includes:

• Internet slang
• Emoji-heavy comments
• Creator terminology
• Sarcasm
• Mixed sentiment
• Short comments
• Viral internet phrases

Results

Metric	Score
Accuracy	88.00%
Macro F1	87.68%
Weighted F1	87.73%

---

📉 Normalized Confusion Matrix

The model performs consistently across all sentiment classes and shows balanced classification behavior. Most classification errors occur between Neutral and Positive comments.

Negative comments are generally identified more reliably.

---

🏋️ Training Details

Base Model

cardiffnlp/twitter-roberta-base-sentiment-latest

Fine-Tuning Dataset

1M+ YouTube Comments

Hardware

NVIDIA RTX 3050 Laptop GPU (6GB)

Training Features

• Mixed Precision Training (AMP)
• Layer-wise Learning Rate Decay (LLRD)
• Gradient Accumulation
• Cosine Learning Rate Scheduler
• Warmup Scheduling
• Gradient Checkpointing
• Class Weighted Loss
• Early Stopping
• Resume Training Support
• Dynamic GPU Configuration

---

📋 Internal Test Set Results

For reference, the held-out test split produced:

Metric	Score
Accuracy	77.43%
Macro F1	77.38%
Weighted F1	77.41%

The external benchmark is considered a better estimate of real-world deployment performance.

---

🔥 Example Predictions

Positive

W video bro 🔥

Prediction:

Positive

---

Positive

Absolute cinema

Prediction:

Positive

---

Negative

nah this ain't it

Prediction:

Negative

---

Negative

Worst update ever

Prediction:

Negative

---

Neutral

Uploaded 2 hours ago

Prediction:

Neutral

---

💻 Usage

from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification
)

MODEL_ID = "rishigupta04/yt-comments-sentiment-analyzer"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID
)

model = AutoModelForSequenceClassification.from_pretrained(
    MODEL_ID
)

---

🎯 Intended Use Cases

Suitable For

• YouTube Comment Analysis
• Social Media Monitoring
• Creator Analytics
• Content Moderation Assistance
• Brand Sentiment Tracking
• NLP Education
• Research Projects

Not Intended For

• Medical Decisions
• Legal Decisions
• Financial Decisions
• High-Risk Automated Systems

---

⚠️ Known Limitations

Like most sentiment models, performance may degrade on:

• Heavy sarcasm
• Irony
• Context-dependent jokes
• Multi-comment conversations
• Ambiguous sentiment

Example:

Wow another genius from YouTube University

May require additional context to classify correctly.

---

🛠️ Project Stack

Python
Transformers
PyTorch
FastAPI
Hugging Face
Scikit-Learn
Pandas
NumPy

---

👨‍💻 Author

Rishiraj Gupta

M.Sc. Data Science

This project demonstrates:

• End-to-End NLP Pipeline
• Transformer Fine-Tuning
• Model Evaluation
• Error Analysis
• MLOps Practices
• FastAPI Deployment
• Hugging Face Deployment
• Real-World Sentiment Analysis

---

⭐ If you find this model useful, please consider liking the repository.

Built with 🤗 Transformers + PyTorch