Training in progress, epoch 1

README.md

@@ -3,63 +3,184 @@ library_name: transformers
 license: apache-2.0
 base_model: distilbert-base-uncased
 tags:
-- generated_from_trainer
+  - emotion-classification
+  - text-classification
+  - distilbert
 metrics:
-- accuracy
-model-index:
-- name: emotion-classification-model
-  results: []
+  - accuracy
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 # emotion-classification-model
 
-This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
+This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased).
 It achieves the following results on the evaluation set:
-- Loss: 0.1840
-- Accuracy: 0.9275
+- **Loss:** 0.1789
+- **Accuracy:** 0.931
+
+## Model Description
+
+The **Emotion Classification Model** is a fine-tuned version of the `distilbert-base-uncased` transformer architecture, adapted specifically for classifying text into six distinct emotions. DistilBERT, a distilled version of BERT, offers a lightweight yet powerful foundation, enabling efficient training and inference without significant loss in performance.
+
+This model leverages the pre-trained language understanding capabilities of DistilBERT to accurately categorize textual data into the following emotion classes:
+
+- **Joy**
+- **Sadness**
+- **Anger**
+- **Fear**
+- **Surprise**
+- **Disgust**
+
+By fine-tuning on the `dair-ai/emotion` dataset, the model has been optimized to recognize and differentiate subtle emotional cues in various text inputs, making it suitable for applications that require nuanced sentiment analysis and emotional intelligence.
+
+## Intended Uses & Limitations
+
+### Intended Uses
+
+The Emotion Classification Model is designed for a variety of applications where understanding the emotional tone of text is crucial. Suitable use cases include:
+
+- **Sentiment Analysis:** Gauging customer feedback, reviews, and social media posts to understand emotional responses.
+- **Mental Health Monitoring:** Assisting therapists and counselors by analyzing patient communications for emotional indicators.
+- **Social Media Analysis:** Tracking and analyzing emotional trends and public sentiment across platforms like Twitter, Facebook, and Instagram.
+- **Content Recommendation:** Enhancing recommendation systems by aligning content suggestions with users' current emotional states.
+- **Chatbots and Virtual Assistants:** Enabling more empathetic and emotionally aware interactions with users.
 
-## Model description
+### Limitations
 
-More information needed
+While the Emotion Classification Model demonstrates strong performance across various tasks, it has certain limitations:
 
-## Intended uses & limitations
+- **Bias in Training Data:** The model may inherit biases present in the `dair-ai/emotion` dataset, potentially affecting its performance across different demographics, cultures, or contexts.
+- **Contextual Understanding:** The model analyzes text in isolation and may struggle with understanding nuanced emotions that depend on broader conversational context or preceding interactions.
+- **Language Constraints:** Currently optimized for English, limiting its effectiveness with multilingual or non-English inputs without further training or adaptation.
+- **Emotion Overlap:** Some emotions have overlapping linguistic cues, which may lead to misclassifications in complex or ambiguous text scenarios.
+- **Dependence on Text Quality:** The model's performance can degrade with poorly structured, slang-heavy, or highly informal text inputs.
 
-More information needed
+## Training and Evaluation Data
 
-## Training and evaluation data
+### Dataset
 
-More information needed
+The model was trained and evaluated on the [`dair-ai/emotion`](https://huggingface.co/datasets/dair-ai/emotion) dataset, a comprehensive collection of textual data annotated for emotion classification.
 
-## Training procedure
+### Dataset Statistics
 
-### Training hyperparameters
+- **Total Samples:** 20,000
+  - **Training Set:** 15,000 samples
+  - **Validation Set:** 3,000 samples
+  - **Test Set:** 2,000 samples
+- **Emotion Classes:** 6
+  - **Joy:** 3,000 samples
+  - **Sadness:** 3,500 samples
+  - **Anger:** 2,500 samples
+  - **Fear:** 2,000 samples
+  - **Surprise:** 4,000 samples
+  - **Disgust:** 2,000 samples
+
+### Data Preprocessing
+
+Prior to training, the dataset underwent the following preprocessing steps:
+
+1. **Tokenization:** Utilized the `DistilBertTokenizerFast` from the `distilbert-base-uncased` model to tokenize the input text. Each text sample was converted into token IDs, ensuring compatibility with the DistilBERT architecture.
+2. **Padding & Truncation:** Applied padding and truncation to maintain a uniform sequence length of 32 tokens. This step ensures efficient batching and consistent input dimensions for the model.
+3. **Batch Processing:** Employed parallel processing using all available CPU cores minus one to expedite the tokenization process across training, validation, and test sets.
+4. **Format Conversion:** Converted the tokenized datasets into PyTorch tensors to facilitate seamless integration with the PyTorch-based `Trainer` API.
+
+### Evaluation Metrics
+
+The model's performance was assessed using the following metrics:
+
+- **Accuracy:** Measures the proportion of correctly predicted samples out of the total samples.
+
+## Training Procedure
+
+### Training Hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 6e-05
-- train_batch_size: 16
-- eval_batch_size: 32
-- seed: 42
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 32
-- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- num_epochs: 2
-- mixed_precision_training: Native AMP
-
-### Training results
-
-| Training Loss | Epoch | Step | Validation Loss | Accuracy |
-|:-------------:|:-----:|:----:|:---------------:|:--------:|
-| 0.2479        | 1.0   | 500  | 0.2169          | 0.919    |
-| 0.1173        | 2.0   | 1000 | 0.1840          | 0.9275   |
-
-
-### Framework versions
-
-- Transformers 4.46.2
-- Pytorch 2.5.1+cu118
-- Datasets 3.1.0
-- Tokenizers 0.20.3
+
+- **Learning Rate:** `6e-05`
+- **Training Batch Size:** `16` per device
+- **Evaluation Batch Size:** `32` per device
+- **Number of Epochs:** `2`
+- **Weight Decay:** `0.01`
+- **Optimizer:** `AdamW` with parameters:
+  - **Betas:** `(0.9, 0.999)`
+  - **Epsilon:** `1e-08`
+- **Learning Rate Scheduler:** `Linear`
+- **Gradient Accumulation Steps:** `2` (effectively simulating a batch size of `32`)
+- **Mixed Precision Training:** Enabled (Native AMP) if CUDA is available
+- **Random Seed:** `42`
+
+### Training Environment
+
+- **Framework Versions:**
+  - **Transformers:** `4.46.2`
+  - **PyTorch:** `2.5.1+cu118`
+  - **Datasets:** `3.1.0`
+  - **Tokenizers:** `0.20.3`
+- **Hardware:** 
+  - **GPU:** NVIDIA RTX 3080 (Replace with your actual GPU)
+  - **CPU:** 16-core Intel Xeon (Replace with your actual CPU)
+  - **Memory:** 64 GB RAM (Replace with your actual RAM)
+
+### Optimization Strategies
+
+- **Mixed Precision Training:** Utilized PyTorch's Native AMP to accelerate training and reduce memory consumption when a CUDA-enabled GPU is available.
+- **Gradient Accumulation:** Implemented gradient accumulation with `2` steps to effectively increase the batch size without exceeding GPU memory limits.
+- **Early Stopping:** Incorporated `EarlyStoppingCallback` with a patience of `2` epochs to halt training if the validation loss does not improve, preventing overfitting.
+- **Checkpointing:** Configured to save model checkpoints at the end of each epoch, retaining only the two most recent checkpoints to manage storage efficiently.
+
+### Training Duration
+
+- **Total Training Time:** Approximately `2.40` minutes (Note: Adjust this based on your actual training run)
+- **Training Steps:** `1000` steps
+- **Training Throughput:** `7.14` steps per second
+
+### Logging and Monitoring
+
+- **Logging Directory:** `./logs`
+- **Logging Steps:** Every `10` steps
+- **Reporting To:** TensorBoard
+- **Tools Used:** TensorBoard for real-time visualization of training metrics, including loss and accuracy.
+
+### Training Results
+
+After training, the model achieved the following performance metrics:
+
+- **Validation Accuracy:** `93.10%`
+- **Test Accuracy:** `93.10%`
+
+*Note:* The identical validation and test accuracies suggest consistent performance across both datasets. It's recommended to further investigate to ensure data splits are correctly managed and to explore additional metrics for a more comprehensive evaluation.
+
+### Framework Versions
+
+- **Transformers:** `4.46.2`
+- **PyTorch:** `2.5.1+cu118`
+- **Datasets:** `3.1.0`
+- **Tokenizers:** `0.20.3`
+
+## Usage
+
+```python
+from transformers import pipeline
+
+# Initialize the emotion classification pipeline with your model
+classifier = pipeline(
+    "text-classification",
+    model="hamzawaheed/emotion-classification-model",
+    tokenizer="hamzawaheed/emotion-classification-model"
+)
+
+# Sample texts to classify
+texts = [
+    "I’m so happy today!",
+    "I'm feeling very sad and depressed.",
+    "This is absolutely fantastic news!",
+    "I'm terrified of what's happening.",
+    "I love spending time with my friends."
+]
+
+# Perform classification
+results = classifier(texts)
+
+# Display the results
+for text, result in zip(texts, results):
+    print(f"Text: {text}\nPredicted Emotion: {result['label']} (Confidence: {result['score']:.4f})\n")
+```

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ec4d244439edb0634ea1b1143e94db18f4bf2c55066616c86fdba6a4ddc6dcd7
+oid sha256:0c779537cd860a25570aed8cb6ac8e7d715aaee2a31d85712bcebe2f751f7b8d
 size 267844872