Instructions to use smd20/imdb-distilbert-sentiment with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use smd20/imdb-distilbert-sentiment with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="smd20/imdb-distilbert-sentiment")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("smd20/imdb-distilbert-sentiment") model = AutoModelForSequenceClassification.from_pretrained("smd20/imdb-distilbert-sentiment", device_map="auto") - Notebooks
- Google Colab
- Kaggle
π¬ IMDb Sentiment Analysis with DistilBERT
A Comparative Study of Embedding-Based and Transformer-Based Architectures for IMDb Sentiment Analysis
This repository contains a fine-tuned DistilBERT model for binary sentiment classification (Positive/Negative) of movie reviews. The model was developed as part of a comparative study evaluating simple neural networks, recurrent networks (BiLSTM), and transformer-based architectures on the IMDB dataset.
π€ Author
Samad Sohrab
π Model Summary
- Base Model: distilbert-base-uncased
- Task: Binary Text Classification (Sentiment Analysis)
- Dataset: IMDB Movie Reviews
- Test Accuracy: 89.95%
- Framework: PyTorch + Hugging Face Transformers
ποΈ Model Architecture
The model is built on top of distilbert-base-uncased with a classification head:
| Component | Details |
|---|---|
| Base Transformer | DistilBERT (6 layers, 768 hidden size, 12 attention heads) |
| Classifier Head | Linear layer (768 β 2) with Dropout |
| Max Token Length | 256 |
| Output | Binary labels (LABEL_0 = Negative, LABEL_1 = Positive) |
π Performance Comparison
The following table compares the proposed DistilBERT model with baseline architectures evaluated in the same study:
| Model | Architecture | Test Accuracy | Final Validation Loss |
|---|---|---|---|
| Baseline | Embedding + GlobalAveragePooling + Dense | 86.04% | 0.323 |
| BiLSTM | Embedding + Bidirectional LSTM + Dense | 85.44% | 0.328 |
| DistilBERT (This Model) | Fine-tuned Transformer | 89.95% | 0.336 |
Key Finding: While recurrent and simple feedforward networks struggle to generalize on IMDB, the transformer-based DistilBERT achieves the highest accuracy, demonstrating the effectiveness of pre-trained language representations.
π How to Use
1. Install Dependencies
pip install transformers torch
2. Load the Model and Predict
from transformers import pipeline classifier = pipeline("text-classification", model="smd20/imdb-distilbert-sentiment") result = classifier("This movie was absolutely fantastic!") print(result)
3. Using PyTorch Directly
import torch from transformers import AutoTokenizer, AutoModelForSequenceClassification model_name = "smd20/imdb-distilbert-sentiment" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSequenceClassification.from_pretrained(model_name) text = "An unforgettable cinematic experience." inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=256) with torch.no_grad(): logits = model(**inputs).logits predicted_class = torch.argmax(logits, dim=-1).item() print(f"Predicted Label: {'Positive' if predicted_class == 1 else 'Negative'}")
ποΈ Training Procedure
Hyperparameters
| Parameter | Value |
|---|---|
| Epochs | 2 |
| Batch Size | 16 |
| Learning Rate | 5e-5 (AdamW) |
| Max Sequence Length | 256 |
| Training Samples | 10,000 |
| Validation Split | 10,000 (from test set used as validation) |
Training Results
| Epoch | Training Loss | Validation Loss | Validation Accuracy |
|---|---|---|---|
| 1 | 0.3598 | 0.2688 | 89.58% |
| 2 | 0.1919 | 0.3362 | 89.95% |
Note: The model shows slight overfitting after epoch 1, suggesting that training for 1 epoch may be sufficient for optimal generalization.
β οΈ Limitations & Bias
- Language: The model is trained on English text only. It may not perform well on non-English reviews.
- Domain: The model is specialized for movie reviews. Performance may degrade on other domains (e.g., product reviews, social media posts).
- Length: Reviews longer than 256 tokens are truncated, which may lose important context.
- Sarcasm & Irony: The model may struggle with sarcastic or ironic statements, as is common in sentiment analysis.
π Citation
If you use this model in your work, please cite:
@misc{sohrab2026imdb, author = {Samad Sohrab}, title = {IMDb Sentiment Analysis with DistilBERT}, year = {2026}, publisher = {Hugging Face}, howpublished = {https://huggingface.co/smd20/imdb-distilbert-sentiment} }
π License
This model is released under the Apache 2.0 license.
- Downloads last month
- -
Model tree for smd20/imdb-distilbert-sentiment
Base model
distilbert/distilbert-base-uncased