README.md

---
library_name: transformers
tags: []
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

Task: Binary Text Classification (Spam vs. Ham).

Dataset: Processed SMS dataset (5,159 samples).

Architecture: (https://huggingface.co/nagaananth/MLOPS_group-v3).

Objective: To accurately identify spam messages while maintaining a low false-positive rate.

Key Feature: The model heavily leverages message length as a discriminative feature, 
as spam messages (avg. ~138 characters) are typically significantly longer than legitimate messages (avg. ~71 characters).

This is the model card of a transformers model that has been pushed on the Hub. 
This model card has been automatically generated.

### Data Overview
Total Samples: 5,159 (after removing 415 duplicates).

Class Distribution: * Label 0 (Ham): 87.5%

Label 1 (Spam): 12.5%

Data Split: * Train: 3,611 samples

Validation: 774 samples

Test: 774 samples

- **Developed by:** [More Information Needed]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** [More Information Needed]

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
- **GitHub Repository:** https://github.com/g25ait2032-prog/MLOPS_Group
- **HF Model:** — v1https://huggingface.co/nagaananth/MLOPS_group-v1
- **HF Model:** — v2 ★ Besthttps://huggingface.co/nagaananth/MLOPS_group-v2
- **HF Model:** — v3https://huggingface.co/nagaananth/MLOPS_group-v3
- **W&B Project Dashboard:** https://wandb.ai/g25ait2032-iit-jodhpur/MLOPS_Group
- **Docker Image (GHCR):** ghcr.io/g25ait2032-prog/mlops_group-inference:latest
- **Kaggle Notebook (v1):** https://www.kaggle.com/code/your-username/sms-spam-v1
- **Kaggle Notebook (v2):** https://www.kaggle.com/code/your-username/sms-spam-v2📓 Kaggle Notebook (v3)https://www.kaggle.com/code/your-username/sms-spam-v3

## Uses

### Direct Use

This model is designed for binary classification of SMS messages into 
"ham" (legitimate) or "spam" (unsolicited marketing/phishing) categories. 
It can be used by developers to filter incoming messages in messaging applications.

### Downstream Use 

The model can be integrated into broader notification filtering systems or used as a 
component in a larger security pipeline to flag suspicious incoming text data for end-users.


### Out-of-Scope Use

This model is not designed for long-form document classification, sentiment analysis, 
or identifying complex conversational nuances. 
It should not be used to automate legal or life-critical decisions 
(e.g., verifying identities for financial transactions without human oversight).

## Bias, Risks, and Limitations
Data Bias: The model is trained on a specific subset of SMS data. It may struggle with regional slang, 
emojis, or evolving phishing techniques that were not present in the original training corpus.  

Risk of False Positives: There is a risk that the model may misclassify important legitimate messages 
(ham) as spam, particularly if they contain keywords frequently associated with spam (e.g., "Urgent," "Click," "Won").  

Contextual Blindness: As a sequence classification model, it processes short text sequences and may lack 
the "memory" or broader conversation context required to understand the intent behind a series of messages.

Phishing Detection: While effective at filtering standard spam, the model may be less reliable at detecting 
highly sophisticated "spear-phishing" attempts that mimic professional language.


### Recommendations

Transparency: Users should be notified when a message is automatically flagged or hidden by this model.

Human-in-the-Loop: We recommend providing an option for users to manually report misclassifications 
so the system can be periodically retuned.


Monitoring: The model’s performance should be monitored for "drift"—as spam tactics change, 
the model's accuracy on newer data may degrade, requiring periodic retraining on current, labeled datasets.

## How to Get Started with the Model
Use the code below to get started with the model.

from transformers import pipeline

# Load your specific model
classifier = pipeline("text-classification", model="your-username/your-model-repo")

# Test with a sample message
print(classifier("URGENT! You have won a 1-week cruise!"))

## Training Details

### Training Data
The model was trained on a curated SMS spam collection. 
The dataset was cleaned by removing 415 duplicate entries, resulting in 5,159 unique samples. 
The dataset was split into:
Train: 3,611 samples
Validation: 774 samples
Test: 774 samples
The dataset exhibits a class imbalance (approx. 87.5% Legitimate / 12.5% Spam), which was accounted for during training.

### Training Procedure
#### Preprocessing
Cleaning: Removal of 415 duplicate messages.

Tokenization: AutoTokenizer for DistilBERT

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

def tokenize(batch):
    return tokenizer(batch["text"], truncation=True, padding="max_length", max_length=128)

train_ds = train_ds.map(tokenize, batched=True)
test_ds  = test_ds.map(tokenize, batched=True)

train_ds = train_ds.rename_column("label", "labels")
test_ds  = test_ds.rename_column("label", "labels")
train_ds.set_format("torch", columns=["input_ids", "attention_mask", "labels"])
test_ds.set_format("torch",  columns=["input_ids", "attention_mask", "labels"])

Labeling: Data was mapped to integers: 0 (Ham) and 1 (Spam).


#### Training Hyperparameters

- **Training regime:** fp32, fp16 mixed precision, bf16 mixed precision,
- bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision
- Optimizer: AdamW.

Learning Rate: 2e-5 (typical for fine-tuning transformers).

Epochs: 3–5 (depending on version; Version 2 converged optimally at 5 epochs).

Batch Size: 16 (or adjusted based on your hardware).

#### Speeds, Sizes, Times 

Average Training Time: ~2 minutes per run.

Infrastructure: Trained on Kaggle environment (T4 x2 GPU or similar).

## Evaluation

Testing Data, Factors & Metrics
Metrics
We used the following metrics to account for class imbalance:

Accuracy: Overall performance.

F1-Score (Weighted/Macro): To evaluate performance on the minority "Spam" class, 
as accuracy alone can be misleading in imbalanced datasets.

Validation Loss: Monitored to prevent overfitting.

### Testing Data, Factors & Metrics

#### Testing Data

The model was evaluated on a held-out test set consisting of 774 samples, ensuring no overlap (zero leakage) 
with the training or validation sets. 
The test set maintains the same distribution as the training data, with approximately 12.4% of 
samples representing the "Spam" class.

#### Factors
The evaluation focuses on the model's ability to distinguish between legitimate messages ("Ham") 
and unsolicited commercial messages ("Spam"). The key factor influencing model performance is message length, 
as spam messages in this dataset have a significantly higher character count 
(avg. ~138 characters) compared to legitimate messages (avg. ~71 characters).

#### Metrics
To handle the class imbalance and ensure reliable performance, we utilized:

Accuracy: Provided as a high-level overview of performance.

F1-Score (Weighted/Macro): Chosen because it balances Precision and Recall, which is crucial 
given that the "Spam" class is the minority class.

Validation Loss: Monitored to identify the point of convergence and detect potential overfitting.

### Results

[More Information Needed]

#### Summary
The model demonstrates exceptional robustness in identifying spam messages. 
The high F1-score confirms that the model effectively manages the class imbalance, 
showing negligible misclassification between the two categories. 
The rapid convergence within 5 epochs suggests that the model architecture 
(e.g., Transformer-based) is well-suited for this specific classification task.


## Model Examination [optional]

Understanding why a model classifies a message as "Spam" versus "Ham" is crucial for building 
trust and ensuring the system isn't relying on irrelevant patterns.

Interpretability Approach
For this Transformer-based model, we can utilize Attention Visualization and Feature Importance techniques:

Attention Mapping: Since Transformer architectures (like BERT or DistilBERT) utilize self-attention mechanisms, 
we can visualize which tokens (words) the model focuses on when making a prediction. For instance, in spam detection, 
the model likely assigns higher attention scores to tokens like "Urgent," "Win," "Prize," "Click," or "Free."

Saliency Maps: These highlight specific words that contributed most significantly to the final classification score. 
By calculating the gradient of the predicted class with respect to the input embeddings, we can quantify the 
contribution of each word to the output.

Interpretability Insights
Preliminary analysis suggests that the model:

Prioritizes Keywords: High-intensity attention is consistently placed on classic spam triggers 
(e.g., promotional urgency or financial incentives).

Captures Length Signals: Given that spam messages in our dataset are on average ~138 characters 
(nearly double that of legitimate messages), the model appears to use message length as a strong secondary heuristic for classification.

Contextual Awareness: Unlike traditional "Bag-of-Words" models, this Transformer captures contextual relationships 
(e.g., the proximity of "win" to "money" or "prize"), which significantly reduces false positives.

## Environmental Impact

Carbon emissions are estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: NVIDIA T4 GPU

Hours used: ~0.03 hours (approx. 2 minutes total training time)

Cloud Provider: Kaggle

Compute Region: US-based data center (approximate)

Carbon Emitted: < 0.01 kg CO₂eq

Note: The carbon footprint for this specific training job is negligible due to the short training 
duration and the efficiency of the model architecture. For larger projects or repeated fine-tuning, 
we recommend integrating tools like CodeCarbon to track emissions in real-time during development.
Carbon emissions are estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

- **Hardware Type:** NVIDIA T4 GPU (Kaggle Standard)
- **Hours used:** ~0.03 hours (approx. 2 minutes total training time)
- **Cloud Provider:** Kaggle (Google Cloud Platform infrastructure)
- **Compute Region:** US (Typically US-Central or US-East for Kaggle)
- **Carbon Emitted:** < 0.01 kg CO₂eq

## Technical Specifications [optional]

### Model Architecture and Objective

Architecture: The model utilizes a Transformer-based architecture (e.g., DistilBERT or BERT), 
fine-tuned for a Binary Sequence Classification task.

Objective: To classify input SMS messages into one of two categories: 0 (Ham/Legitimate) or 1 (Spam).

Mechanism: The model leverages self-attention layers to identify contextual patterns associated with spam 
(e.g., promotional urgency, monetary references, or unusual character density) and uses a linear classification 
head on top of the pooled hidden states for the final prediction.

### Compute Infrastructure

#### Hardware

Environment: Kaggle Notebooks.

Accelerator: NVIDIA T4 GPU (used for accelerated fine-tuning and inference).

#### Software

Framework: PyTorch and Hugging Face Transformers library.

Optimization: fp16 mixed-precision training was used to reduce memory consumption and accelerate training 
time without compromising model accuracy.

Libraries: datasets, transformers, evaluate, and accelerate.

## Citation [optional]


**BibTeX:**

@misc{sms-spam-classifier-2026,
  author = {Your Name},
  title = {SMS Spam Classifier: A Fine-tuned Transformer Model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/your-username/your-model-repo}}
}


**APA:**

Duggirala Vnaga Ananth. (2026). SMS Spam Classifier: A Fine-tuned Transformer Model [Computer model]. 
https://huggingface.co/nagaananth/MLOPS_group-v1/

## Glossary [optional]

Ham: A common term used in spam filtering to denote legitimate, non-spam messages.

Spam: Unsolicited or unwanted commercial messages.

Transformer: A deep learning architecture that uses self-attention mechanisms to 
weigh the significance of different parts of the input data.

F1-Score: A metric that balances precision and recall; highly useful for evaluating models 
on imbalanced datasets where one class is much more frequent than the other.

Fine-tuning: The process of taking a pre-trained language model and training it further on a smaller, 
task-specific dataset.

## More Information [optional]

This model was developed to provide a lightweight and efficient solution for SMS spam filtering. 
By leveraging transfer learning, the model achieves high accuracy with minimal training time, 
making it suitable for deployment in resource-constrained environments.

## Model Card Authors [optional]

G25AIT2032 Duggirala Vnaga Ananth

## Model Card Contact

For questions or feedback regarding this model, please reach out via:

GitHub: https://github.com/g25ait2032-prog/MLOPS_Group

Hugging Face: https://huggingface.co/nagaananth/MLOPS_group-v1

Email: g25ait2032@iitj.ac.in