Update README.md

README.md

@@ -1,199 +1,258 @@
 ---
 library_name: transformers
-tags: []
+tags:
+  - text-classification
+  - spam-detection
+  - distilbert
+  - sms
+  - pytorch
+language:
+  - en
+license: mit
+datasets:
+  - sms_spam
+metrics:
+  - accuracy
+  - f1
+pipeline_tag: text-classification
 ---
 
-# Model Card for Model ID
+# SMS Spam Classifier — DistilBERT (Group 36, IIT Jodhpur)
 
-<!-- Provide a quick summary of what the model is/does. -->
+Fine-tuned `distilbert-base-uncased` for binary SMS spam classification. Achieves **99.35% accuracy** and **0.9851 F1 Macro** on the held-out test set. This is **v2** — the best-performing version by validation loss (0.0292).
 
+Developed as part of the MLOps course, PGD AI Program, IIT Jodhpur.
 
+---
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+- **Base model:** `distilbert-base-uncased` (66M parameters)
+- **Task:** Binary text classification — Ham (0) vs Spam (1)
+- **Dataset:** UCI SMS Spam Collection (5,159 samples after deduplication)
+- **Architecture:** DistilBERT encoder + linear classification head
+- **Framework:** PyTorch + Hugging Face Transformers
+- **Training platform:** Kaggle (NVIDIA T4 x2 GPU)
+- **Developed by:** MLOps Group 36, IIT Jodhpur
+- **Model card authors:** G25AIT2032 Duggirala Vnaga Ananth
+- **Contact:** g25ait2032@iitj.ac.in
+
+### Related Resources
+
+| Resource | Link |
+|---|---|
+| GitHub Repository | [github.com/g25ait2032-prog/mlops-group36-iitj](https://github.com/g25ait2032-prog/mlops-group36-iitj) |
+| W&B Experiment Dashboard | [wandb.ai/g25ait2032-iit-jodhpur/MLOPS_Group](https://wandb.ai/g25ait2032-iit-jodhpur/MLOPS_Group) |
+| HF Model — v1 | [nagaananth/MLOPS_group-v1](https://huggingface.co/nagaananth/MLOPS_group-v1) |
+| HF Model — v2 ★ Best | [nagaananth/MLOPS_group-v2](https://huggingface.co/nagaananth/MLOPS_group-v2) |
+| HF Model — v3 | [nagaananth/MLOPS_group-v3](https://huggingface.co/nagaananth/MLOPS_group-v3) |
+| HF Model — v4 | [nagaananth/MLOPS_group-v4](https://huggingface.co/nagaananth/MLOPS_group-v4) |
+| Docker Image (GHCR) | `ghcr.io/g25ait2032-prog/mlops_group-inference:latest` |
+| Docker Image (Hub) | `dvnananth/mlops-group36:v1` |
 
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+---
 
-[More Information Needed]
+## How to Get Started
 
-### Out-of-Scope Use
+```python
+from transformers import pipeline
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+classifier = pipeline(
+    "text-classification",
+    model="nagaananth/MLOPS_group-v2"
+)
 
-[More Information Needed]
+# Spam example
+print(classifier("URGENT! You have won a free iPhone. Click here now."))
+# [{'label': 'spam', 'score': 0.9804}]
 
-## Bias, Risks, and Limitations
+# Ham example
+print(classifier("Hey, are we still meeting for lunch at 12?"))
+# [{'label': 'ham', 'score': 0.9982}]
+```
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+Or with full control:
 
-[More Information Needed]
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
 
-### Recommendations
+model_name = "nagaananth/MLOPS_group-v2"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+model.eval()
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+def predict(text):
+    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
+    with torch.no_grad():
+        logits = model(**inputs).logits
+    probs = torch.softmax(logits, dim=-1)[0]
+    pred_idx = probs.argmax().item()
+    return {
+        "label": model.config.id2label[pred_idx],
+        "score": round(probs[pred_idx].item(), 4)
+    }
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+print(predict("Free prize! Click now to claim your reward."))
+# {'label': 'spam', 'score': 0.9897}
+```
 
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
+---
 
 ## Training Details
 
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+### Dataset
 
-### Training Procedure
+**UCI SMS Spam Collection** loaded via HuggingFace `datasets` (`sms_spam`).
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+| Split | Samples | Ham % | Spam % |
+|---|---|---|---|
+| Train (70%) | 3,611 | ~87.5 | ~12.5 |
+| Validation (15%) | 774 | ~87.5 | ~12.5 |
+| Test (15%) | 774 | ~87.5 | ~12.5 |
 
-#### Preprocessing [optional]
+**Preprocessing steps:**
+- Lowercased and whitespace normalised
+- 415 duplicate messages removed (total: 5,159 unique samples)
+- Stratified 70/15/15 split with zero-leakage verification
+- Tokenized with `AutoTokenizer` for DistilBERT (`truncation=True, max_length=128`)
+- Labels mapped: `{"ham": 0, "spam": 1}`
 
-[More Information Needed]
+### Hyperparameter Comparison (All Versions)
 
+| Version | LR | Epochs | Batch Size | Warmup | Weight Decay | Early Stopping | Val Loss | F1 Macro |
+|---|---|---|---|---|---|---|---|---|
+| v1 | 3e-5 | 3 | 16 | 100 | 0.01 | No | 0.0539 | 0.9849 |
+| v2 ★ | 2e-5 | 5 | 32 | 200 | 0.01 | Yes (p=2) | **0.0292** | **0.9851** |
+| v3 | 2e-5 | 5 | 32 | 200 | 0.01 | Yes (p=2) | 0.0376 | 0.9851 |
+| v4 | 1e-5 | 4 | 16 | 200 | 0.02 | Yes (p=2) | — | — |
 
-#### Training Hyperparameters
+**v2** was selected as the final deployment model due to its lowest validation loss (0.0292), indicating the best generalisation.
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+### Training Configuration (v2)
 
-#### Speeds, Sizes, Times [optional]
+- **Optimizer:** AdamW
+- **Learning rate:** 2e-5
+- **Epochs:** 5 (with early stopping, patience=2)
+- **Batch size:** 32 (train), 64 (eval)
+- **Mixed precision:** fp16
+- **Metric for best model:** F1 Weighted
+- **Infrastructure:** Kaggle NVIDIA T4 x2 GPU
+- **Average training time:** ~2 minutes per run
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+---
 
-[More Information Needed]
+## Evaluation Results
 
-## Evaluation
+### Test Set Performance (v2 — Best Model)
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+| Metric | Score |
+|---|---|
+| Accuracy | 0.9935 |
+| F1 Weighted | 0.9935 |
+| F1 Macro | 0.9851 |
+| Precision | 0.9935 |
+| Recall | 0.9935 |
+| Validation Loss | 0.0292 |
 
-### Testing Data, Factors & Metrics
+### Adversarial Test Cases
 
-#### Testing Data
+The model was evaluated on 15 adversarial/edge-case SMS messages covering spam, ham, and ambiguous phrasing (e.g., messages mixing casual language with spam triggers). Representative examples:
 
-<!-- This should link to a Dataset Card if possible. -->
+| Text | True | Predicted | Confidence |
+|---|---|---|---|
+| "URGENT! You have won a 1-week cruise! Call now." | spam | spam | 0.9987 |
+| "You won! Click here to claim your prize." | spam | spam | 0.9945 |
+| "Hey, are we still meeting for lunch at 12?" | ham | ham | 0.9991 |
+| "Can you send me the report by EOD?" | ham | ham | 0.9988 |
+| "Meeting for lunch? I won a contest, let's talk." | ham | ham | 0.9756 |
 
-[More Information Needed]
+### Inference Latency (CPU)
 
-#### Factors
+- Mean latency: ~30–60 ms per sample
+- Suitable for CPU-only deployment
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+---
 
-[More Information Needed]
+## Uses
 
-#### Metrics
+### Direct Use
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+Binary classification of SMS or short-text messages into `ham` (legitimate) or `spam` (unsolicited/phishing). Can be directly integrated into messaging applications or notification pipelines.
 
-[More Information Needed]
+### Downstream Use
 
-### Results
+Can serve as a component in broader security pipelines for filtering suspicious incoming messages, or as a baseline for transfer learning to other spam-detection domains.
 
-[More Information Needed]
+### Out-of-Scope Use
 
-#### Summary
+- Long-form document classification
+- Sentiment analysis or intent detection
+- Legal or financial decision-making without human oversight
+- Languages other than English
 
+---
 
+## Bias, Risks, and Limitations
 
-## Model Examination [optional]
+**Data Bias:** Trained on a specific SMS corpus from the early 2010s. May struggle with modern slang, emojis, or evolved phishing techniques not present in the training data.
 
-<!-- Relevant interpretability work for the model goes here -->
+**False Positives:** Messages containing spam-adjacent keywords (e.g., "Urgent", "Click", "Won") in legitimate contexts may be misclassified.
 
-[More Information Needed]
+**Contextual Blindness:** Processes each message independently; cannot use conversational context from prior messages.
 
-## Environmental Impact
+**Phishing Sophistication:** Less reliable against highly sophisticated spear-phishing that mimics professional language.
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+### Recommendations
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- Notify users when a message is flagged automatically.
+- Provide a manual override/report mechanism for misclassifications.
+- Monitor for distribution drift and retrain periodically on newer data.
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+---
 
-## Technical Specifications [optional]
+## Technical Specifications
 
-### Model Architecture and Objective
+### Model Architecture
 
-[More Information Needed]
+- **Base:** `distilbert-base-uncased` (6 transformer layers, 768 hidden dim, 12 attention heads)
+- **Classification head:** Linear layer over `[CLS]` token pooled output → 2 classes
+- **Total parameters:** ~66M
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
+- **Training:** Kaggle Notebooks — NVIDIA T4 x2 GPU
+- **Libraries:** `transformers`, `datasets`, `evaluate`, `accelerate`, `torch`, `wandb`
+- **Inference:** CPU-compatible (no GPU required)
 
-#### Software
+### Environmental Impact
 
-[More Information Needed]
+- **Hardware:** NVIDIA T4 GPU (Kaggle)
+- **Training duration:** ~2 minutes per run
+- **Carbon emitted:** < 0.01 kg CO₂eq (estimated via ML Impact Calculator)
 
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
+---
 
-[More Information Needed]
+## Citation
 
-## Model Card Authors [optional]
+```bibtex
+@misc{group36-sms-spam-2026,
+  author    = {Duggirala Vnaga Ananth and Anukumar K and Shrikrishna Tripathi and Sudeb Ghosh},
+  title     = {SMS Spam Classifier: Fine-tuned DistilBERT (Group 36, IIT Jodhpur)},
+  year      = {2026},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/nagaananth/MLOPS_group-v2}}
+}
+```
 
-[More Information Needed]
+---
 
-## Model Card Contact
+## Glossary
 
-[More Information Needed]
+- **Ham:** Legitimate, non-spam SMS message
+- **Spam:** Unsolicited commercial or phishing message
+- **DistilBERT:** Distilled version of BERT — 40% smaller, retains 97% of BERT's NLU performance
+- **F1 Macro:** Unweighted mean of per-class F1 scores; useful for evaluating imbalanced datasets
+- **Fine-tuning:** Adapting a pre-trained language model to a task-specific dataset with supervised training