|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- text-classification |
|
|
- spam-detection |
|
|
- sms |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# 🛡️ Model Card for `alusci/distilbert-smsafe` |
|
|
|
|
|
A lightweight DistilBERT model fine-tuned for spam detection in SMS messages. The model classifies input messages as either **spam** or **ham** (not spam), using a custom dataset of real-world OTP (One-Time Password) and spam SMS messages. |
|
|
|
|
|
--- |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
- **Developed by:** [alusci](https://huggingface.co/alusci) |
|
|
- **Model type:** Transformer-based binary classifier |
|
|
- **Language(s):** English |
|
|
- **License:** Apache 2.0 |
|
|
- **Finetuned from model:** `distilbert-base-uncased` |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Repository:** [https://huggingface.co/alusci/distilbert-smsafe](https://huggingface.co/alusci/distilbert-smsafe) |
|
|
|
|
|
--- |
|
|
|
|
|
## 🛠️ Uses |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
- Detect whether an SMS message is spam or ham (OTP or not). |
|
|
- Useful in prototypes, educational settings, or lightweight filtering applications. |
|
|
|
|
|
```python |
|
|
from transformers import pipeline |
|
|
|
|
|
classifier = pipeline("text-classification", model="alusci/distilbert-smsafe") |
|
|
result = classifier("Your verification code is 123456. Please do not share it with anyone.") |
|
|
|
|
|
# Optional: map the label to human-readable terms |
|
|
label_map = {"LABEL_0": "ham", "LABEL_1": "spam"} |
|
|
print(f"Label: {label_map[result[0]['label']]} - Score: {result[0]['score']:.2f}") |
|
|
``` |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
- Not intended for email spam detection or multilingual message filtering. |
|
|
- Not suitable for production environments without further testing and evaluation. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧪 Bias, Risks, and Limitations |
|
|
|
|
|
- The model may reflect dataset biases (e.g., message structure, language patterns). |
|
|
- It may misclassify legitimate OTPs or non-standard spam content. |
|
|
- Risk of false positives in edge cases. |
|
|
|
|
|
### Recommendations |
|
|
|
|
|
- Evaluate on your own SMS dataset before deployment. |
|
|
- Consider combining with rule-based or heuristic systems in production. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
- Dataset used: [`alusci/sms-otp-spam-dataset`](https://huggingface.co/datasets/alusci/sms-otp-spam-dataset) |
|
|
- Binary labels for spam and non-spam OTP messages |
|
|
|
|
|
### Training Procedure |
|
|
|
|
|
- **Epochs:** 5 |
|
|
- **Batch Size:** 16 (assumed) |
|
|
- **Loss Function:** CrossEntropyLoss |
|
|
- **Optimizer:** AdamW |
|
|
- **Tokenizer:** `distilbert-base-uncased` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📈 Evaluation |
|
|
|
|
|
### Metrics |
|
|
|
|
|
- Accuracy, Precision, Recall, F1-score on held-out validation set |
|
|
- Binary classification labels: |
|
|
- `LABEL_0` → ham |
|
|
- `LABEL_1` → spam |
|
|
|
|
|
### Results |
|
|
|
|
|
**Evaluation metrics after 5 epochs:** |
|
|
|
|
|
- **Loss:** 0.2962 |
|
|
- **Accuracy:** 91.35% |
|
|
- **Precision:** 90.26% |
|
|
- **Recall:** 100.00% |
|
|
- **F1-score:** 94.88% |
|
|
|
|
|
**Performance:** |
|
|
|
|
|
- **Evaluation runtime:** 4.37 seconds |
|
|
- **Samples/sec:** 457.27 |
|
|
- **Steps/sec:** 9.15 |
|
|
|
|
|
--- |
|
|
|
|
|
## 🌱 Environmental Impact |
|
|
|
|
|
- **Hardware Type:** Apple Silicon MPS GPU (Mac) |
|
|
- **Hours used:** <1 hour (small dataset) |
|
|
- **Cloud Provider:** None (trained locally) |
|
|
- **Carbon Emitted:** Minimal due to local and efficient hardware |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔧 Technical Specifications |
|
|
|
|
|
### Model Architecture and Objective |
|
|
|
|
|
- **Base:** DistilBERT |
|
|
- **Objective:** Binary classification head on pooled output |
|
|
- **Parameters:** ~66M (same as distilbert) |
|
|
|
|
|
--- |
|
|
|
|
|
## 📬 Model Card Contact |
|
|
|
|
|
For questions or feedback, please contact via [Hugging Face profile](https://huggingface.co/alusci). |
