---
language: en
license: mit
library_name: peft
tags:
- sentiment-analysis
- lora
- distilbert
- peft
- amazon-reviews
datasets:
- amazon_polarity
metrics:
- accuracy
- f1
model-index:
- name: distilbert-lora-sentiment-amazon
  results:
  - task:
      type: text-classification
      name: Sentiment Analysis
    dataset:
      name: Amazon Polarity
      type: amazon_polarity
      split: test
    metrics:
    - type: accuracy
      value: 0.878
      name: Accuracy
    - type: f1
      value: 0.878
      name: F1 Score
widget:
- text: "This product is absolutely amazing! I love it."
  example_title: "Positive Review"
- text: "Worst purchase ever. Completely useless."
  example_title: "Negative Review"
- text: "Good product but shipping was slow."
  example_title: "Mixed Review"
---

# DistilBERT-LoRA Sentiment Classifier (Amazon Reviews)

## Model Description

**Developed by:** sunil9938  
**Model type:** Text Classification (Sentiment Analysis)  
**Language:** English  
**License:** MIT  
**Finetuned from model:** distilbert-base-uncased

This model is a fine-tuned version of **DistilBERT** using **LoRA (Low-Rank Adaptation)** for sentiment analysis on Amazon product reviews. It achieves **87.8% accuracy** on the test set with only **1.09% trainable parameters**.

### Key Features
- Efficient: Only 739,586 trainable parameters
- Accurate: 87.8% accuracy
- Fast: ~50ms inference time on CPU
- Lightweight: LoRA adapter is only 2.96 MB

## How to Use

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel

tokenizer = AutoTokenizer.from_pretrained("sunil9938/distilbert-lora-sentiment-amazon")
base_model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased",
    num_labels=2,
    ignore_mismatched_sizes=True
)
model = PeftModel.from_pretrained(base_model, "sunil9938/distilbert-lora-sentiment-amazon")
model.eval()

def predict(text):
    inputs = tokenizer(text, truncation=True, padding=True, max_length=256, return_tensors="pt")
    outputs = model(**inputs)
    probs = outputs.logits.softmax(dim=1)
    pred = probs.argmax().item()
    return "POSITIVE" if pred == 1 else "NEGATIVE", probs[0][pred].item()

print(predict("This product is amazing!"))



## Limitations

- English only
- Binary classification (no neutral)
- Trained on Amazon reviews only

## Citation

```bibtex
@misc{sunil9938-distilbert-lora-sentiment,
  author = {Sunil Kumar},
  title = {DistilBERT-LoRA Sentiment Classifier for Amazon Reviews},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/sunil9938/distilbert-lora-sentiment-amazon}
}