Training in progress, epoch 1

README.md

@@ -3,127 +3,61 @@ library_name: transformers
 license: apache-2.0
 base_model: distilbert-base-uncased
 tags:
-  - emotion-classification
-  - text-classification
-  - distilbert
+- generated_from_trainer
 metrics:
-  - accuracy
+- accuracy
+model-index:
+- name: emotion-classification-model
+  results: []
 ---
 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
 # emotion-classification-model
 
-This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased).
+This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- **Loss:** 0.1789
-- **Accuracy:** 0.931
-
-## Model Description
-
-The **Emotion Classification Model** is a fine-tuned version of the `distilbert-base-uncased` transformer architecture, adapted specifically for classifying text into six distinct emotions. DistilBERT, a distilled version of BERT, offers a lightweight yet powerful foundation, enabling efficient training and inference without significant loss in performance.
-
-This model leverages the pre-trained language understanding capabilities of DistilBERT to accurately categorize textual data into the following emotion classes:
-
-- **Joy**
-- **Sadness**
-- **Anger**
-- **Fear**
-- **Surprise**
-- **Disgust**
-
-By fine-tuning on the `dair-ai/emotion` dataset, the model has been optimized to recognize and differentiate subtle emotional cues in various text inputs, making it suitable for applications that require nuanced sentiment analysis and emotional intelligence.
-
-## Intended Uses & Limitations
-
-### Intended Uses
-
-The Emotion Classification Model is designed for a variety of applications where understanding the emotional tone of text is crucial. Suitable use cases include:
-
-- **Sentiment Analysis:** Gauging customer feedback, reviews, and social media posts to understand emotional responses.
-- **Mental Health Monitoring:** Assisting therapists and counselors by analyzing patient communications for emotional indicators.
-- **Social Media Analysis:** Tracking and analyzing emotional trends and public sentiment across platforms like Twitter, Facebook, and Instagram.
-- **Content Recommendation:** Enhancing recommendation systems by aligning content suggestions with users' current emotional states.
-- **Chatbots and Virtual Assistants:** Enabling more empathetic and emotionally aware interactions with users.
-
-### Limitations
+- Loss: 0.1953
+- Accuracy: 0.9225
 
-While the Emotion Classification Model demonstrates strong performance across various tasks, it has certain limitations:
+## Model description
 
-- **Bias in Training Data:** The model may inherit biases present in the `dair-ai/emotion` dataset, potentially affecting its performance across different demographics, cultures, or contexts.
-- **Contextual Understanding:** The model analyzes text in isolation and may struggle with understanding nuanced emotions that depend on broader conversational context or preceding interactions.
-- **Language Constraints:** Currently optimized for English, limiting its effectiveness with multilingual or non-English inputs without further training or adaptation.
-- **Emotion Overlap:** Some emotions have overlapping linguistic cues, which may lead to misclassifications in complex or ambiguous text scenarios.
-- **Dependence on Text Quality:** The model's performance can degrade with poorly structured, slang-heavy, or highly informal text inputs.
+More information needed
 
-## Training and Evaluation Data
+## Intended uses & limitations
 
-### Dataset
+More information needed
 
-The model was trained and evaluated on the [`dair-ai/emotion`](https://huggingface.co/datasets/dair-ai/emotion) dataset, a comprehensive collection of textual data annotated for emotion classification.
+## Training and evaluation data
 
-### Dataset Statistics
+More information needed
 
-- **Total Samples:** 20,000
-  - **Training Set:** 15,000 samples
-  - **Validation Set:** 2,000 samples
-  - **Test Set:** 2,000 samples
-- **Emotion Classes:** 6
-  - **Joy:** 3,000 samples
-  - **Sadness:** 3,500 samples
-  - **Anger:** 2,500 samples
-  - **Fear:** 2,000 samples
-  - **Surprise:** 4,000 samples
-  - **Disgust:** 2,000 samples
+## Training procedure
 
-### Data Preprocessing
-
-Prior to training, the dataset underwent the following preprocessing steps:
-
-1. **Tokenization:** Utilized the `DistilBertTokenizerFast` from the `distilbert-base-uncased` model to tokenize the input text. Each text sample was converted into token IDs, ensuring compatibility with the DistilBERT architecture.
-2. **Padding & Truncation:** Applied padding and truncation to maintain a uniform sequence length of 32 tokens. This step ensures efficient batching and consistent input dimensions for the model.
-3. **Batch Processing:** Employed parallel processing using all available CPU cores minus one to expedite the tokenization process across training, validation, and test sets.
-4. **Format Conversion:** Converted the tokenized datasets into PyTorch tensors to facilitate seamless integration with the PyTorch-based `Trainer` API.
-
-### Evaluation Metrics
-
-The model's performance was assessed using the following metrics:
-
-- **Accuracy:** Measures the proportion of correctly predicted samples out of the total samples.
-
-## Training Procedure
-
-### Training Hyperparameters
+### Training hyperparameters
 
 The following hyperparameters were used during training:
+- learning_rate: 6e-05
+- train_batch_size: 16
+- eval_batch_size: 32
+- total_train_batch_size: 32
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- num_epochs: 2
+- mixed_precision_training: Native AMP
 
-- **Learning Rate:** `6e-05`
-- **Training Batch Size:** `16` per device
-- **Evaluation Batch Size:** `32` per device
-- **Number of Epochs:** `2`
-- **Weight Decay:** `0.01`
-- **Learning Rate Scheduler:** `Linear`
-- **Gradient Accumulation Steps:** `2` (effectively simulating a batch size of `32`)
-- **Mixed Precision Training:** Enabled (Native AMP) if CUDA is available
-
-### Optimization Strategies
-
-- **Mixed Precision Training:** Utilized PyTorch's Native AMP to accelerate training and reduce memory consumption when a CUDA-enabled GPU is available.
-- **Gradient Accumulation:** Implemented gradient accumulation with `2` steps to effectively increase the batch size without exceeding GPU memory limits.
-- **Early Stopping:** Incorporated `EarlyStoppingCallback` with a patience of `2` epochs to halt training if the validation loss does not improve, preventing overfitting.
-- **Checkpointing:** Configured to save model checkpoints at the end of each epoch, retaining only the two most recent checkpoints to manage storage efficiently.
-
-### Training Duration
-
-- **Total Training Time:** Approximately `2.40` minutes
-### Logging and Monitoring
+### Training results
 
-- **Logging Directory:** `./logs`
-- **Logging Steps:** Every `10` steps
-- **Reporting To:** TensorBoard
-- **Tools Used:** TensorBoard for real-time visualization of training metrics, including loss and accuracy.
+| Training Loss | Epoch | Step | Validation Loss | Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|
+| 0.2344        | 1.0   | 500  | 0.2365          | 0.915    |
+| 0.1408        | 2.0   | 1000 | 0.1953          | 0.9225   |
 
-### Training Results
 
-After training, the model achieved the following performance metrics:
+### Framework versions
 
-- **Validation Accuracy:** `93.10%`
-- **Test Accuracy:** `93.10%`
+- Transformers 4.46.2
+- Pytorch 2.5.1+cu118
+- Datasets 3.1.0
+- Tokenizers 0.20.3

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:19d1a3c44772e96b5327f31056e7ac16e6cefd360b89807853755221ba43d3a5
+oid sha256:27aacc44bdf0c1dab0cecec322b3ac730f3600530ca2c309e16a6f1080cd4020
 size 267844872

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d1f16fa5596c73967f5afc8c25e593f01bfb06ae91ebe31f9157c543ed6f4ef1
-size 5240
+oid sha256:037b20a108e5e1bbfc84287b9c78f6ee884f750423b982e7559a78a0b4e5e6e6
+size 5304