ranjana1811/fine-tuned-distilbert-lora

Fine-tuned DistilBERT with LoRA for Sentiment Classification

This model is a lightweight DistilBERT fine-tuned with LoRA adapters on the IMDB movie-review dataset.
It predicts whether a given review is Positive or Negative and is used inside the Self-Healing Classification Pipeline project.

---

Model Details

• Base model: distilbert-base-uncased
• Fine-tuning method: LoRA (Low-Rank Adaptation)
• Dataset: IMDB (50 k movie reviews)
• Labels: 0 = Negative, 1 = Positive
• Framework: 🤗Transformers / PEFT
• Trained in: Google Colab GPU

---

Usage Example

from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("ranjana1811/fine-tuned-distilbert-lora")
model = AutoModelForSequenceClassification.from_pretrained("ranjana1811/fine-tuned-distilbert-lora")

text = "I loved every moment, the acting was superb!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
pred = outputs.logits.softmax(dim=-1).argmax().item()
print(["Negative","Positive"][pred])

---

Intended Use

• Sentiment analysis for short texts or movie reviews.
• Educational demo for explainable AI and LangGraph workflows.
• Designed to be lightweight and run easily on CPU.

Not intended for: hate-speech detection, multi-label, or domain-specific tasks beyond simple sentiment.

---

Training Summary

Parameter	Value
Epochs	3
Batch Size	16
Learning Rate	2e-5
Max Seq Len	256
LoRA Rank (r)	8
LoRA Alpha	16

Model trained with Trainer API on a small IMDB subset for faster Colab runs.

---

Evaluation (Validation subset)

Metric	Score
Accuracy	≈ 0.90
F1 Score	≈ 0.90

---

Limitations & Risks

• Can be over-confident on very short texts.
• Works best on English movie reviews; not tested on other domains.
• Does not filter biased or offensive content.

---

Environmental Note

Trained for ≈ 20 minutes on a T4 GPU in Google Colab – low carbon footprint.

---

Project Links

• Fine-tuning Notebook: Open in Colab
• Source Code Repo: GitHub – Self-Healing Classifier

---

Author

Ranjana ( @ranjana1811 ) For internship submission – ATG Machine Learning Assignment 2025.

---