metadata
language: en
license: mit
library_name: peft
tags:
- sentiment-analysis
- lora
- distilbert
- peft
- amazon-reviews
datasets:
- amazon_polarity
metrics:
- accuracy
- f1
model-index:
- name: distilbert-lora-sentiment-amazon
results:
- task:
type: text-classification
name: Sentiment Analysis
dataset:
name: Amazon Polarity
type: amazon_polarity
split: test
metrics:
- type: accuracy
value: 0.878
name: Accuracy
- type: f1
value: 0.878
name: F1 Score
widget:
- text: This product is absolutely amazing! I love it.
example_title: Positive Review
- text: Worst purchase ever. Completely useless.
example_title: Negative Review
- text: Good product but shipping was slow.
example_title: Mixed Review
DistilBERT-LoRA Sentiment Classifier (Amazon Reviews)
Model Description
Developed by: sunil9938
Model type: Text Classification (Sentiment Analysis)
Language: English
License: MIT
Finetuned from model: distilbert-base-uncased
This model is a fine-tuned version of DistilBERT using LoRA (Low-Rank Adaptation) for sentiment analysis on Amazon product reviews. It achieves 87.8% accuracy on the test set with only 1.09% trainable parameters.
Key Features
- Efficient: Only 739,586 trainable parameters
- Accurate: 87.8% accuracy
- Fast: ~50ms inference time on CPU
- Lightweight: LoRA adapter is only 2.96 MB
How to Use
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel
tokenizer = AutoTokenizer.from_pretrained("sunil9938/distilbert-lora-sentiment-amazon")
base_model = AutoModelForSequenceClassification.from_pretrained(
"distilbert-base-uncased",
num_labels=2,
ignore_mismatched_sizes=True
)
model = PeftModel.from_pretrained(base_model, "sunil9938/distilbert-lora-sentiment-amazon")
model.eval()
def predict(text):
inputs = tokenizer(text, truncation=True, padding=True, max_length=256, return_tensors="pt")
outputs = model(**inputs)
probs = outputs.logits.softmax(dim=1)
pred = probs.argmax().item()
return "POSITIVE" if pred == 1 else "NEGATIVE", probs[0][pred].item()
print(predict("This product is amazing!"))
## Limitations
- English only
- Binary classification (no neutral)
- Trained on Amazon reviews only
## Citation
```bibtex
@misc{sunil9938-distilbert-lora-sentiment,
author = {Sunil Kumar},
title = {DistilBERT-LoRA Sentiment Classifier for Amazon Reviews},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/sunil9938/distilbert-lora-sentiment-amazon}
}