Update README.md

README.md

@@ -9,6 +9,86 @@ pipeline_tag: text-classification
 
 This is the repo for Gen AI final project
 
+# Transformer with Emotion Classification
+
+## Overview
+
+This is a Transformer-based model designed for **emotion classification** and **dialogue act recognition** on the [DailyDialog](http://yanran.li/dailydialog) dataset. It processes multi-turn dialogues to predict emotional states and communication intentions. A **Stacked Autoencoder (SAE)** is included to regularize node usage, encouraging sparsity in feature representations.
+
+While the model successfully predicts dialogue acts, it faces challenges in emotion classification, often outputting binary labels (0 or 1) due to imbalanced data or other limitations.
+
+---
+
+## Model Details
+
+### Model Architecture
+- **Transformer Encoder**: A standard Transformer encoder serves as the backbone for extracting contextual features from dialogues.
+- **Batch Normalization**: Applied to normalize extracted features.
+- **Dropout**: Used to reduce overfitting.
+- **Stacked Autoencoder (SAE)**: Regularizes feature representations by encouraging sparsity, adding KL divergence loss during training.
+- **Classification Heads**:
+  - **Dialogue Act Classifier**: Predicts communication intentions (e.g., inform, question).
+  - **Emotion Classifier**: Predicts one of the annotated emotions (e.g., happiness, sadness, anger).
+
+### Input
+- **input_ids**: Tokenized input sequences of dialogues.
+- **attention_mask**: Binary mask indicating valid tokens in the input sequence.
+
+### Output
+- **act_output**: Predicted dialogue act class.
+- **emotion_output**: Predicted emotion class.
+- **kl_div**: KL divergence loss for SAE regularization.
+
+---
+
+## Dataset: DailyDialog
+
+The model is trained and evaluated on the [DailyDialog](http://yanran.li/dailydialog) dataset.  
+
+### Dataset Features
+- **Size**: 13,118 dialogues with an average of 8 turns per dialogue.
+- **Annotations**:
+  - **Dialogue Acts**: Intentions like inform, question, directive, and commissive.
+  - **Emotions**: Labels such as happiness, sadness, anger, surprise, and no emotion.
+- **License**: [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/).
+
+---
+
+## Usage
+
+### Training
+1. **Dataset Preparation**: Use tokenizers to preprocess the DailyDialog dataset into `input_ids` and `attention_mask`.
+2. **Training Steps**:
+   - Forward pass the input through the model.
+   - Compute cross-entropy loss for the dialogue act and emotion classifiers.
+   - Add KL divergence loss for SAE regularization.
+   - Backpropagate and update parameters.
+
+### Inference
+- Input: Tokenized text sequences and attention masks.
+- Output: Predicted dialogue acts and emotion classes.
+
+---
+
+## Limitations
+- **Emotion Classification**: The model struggles to predict diverse emotional states, often outputting binary values (0 or 1).
+- **Imbalanced Dataset**: Emotion labels in the DailyDialog dataset are not evenly distributed, which impacts model performance.
+- **Limited Domain**: The dataset is focused on daily conversations, so the model may not generalize well to other dialogue contexts.
+
+---
+
+## Citation
+
+If you use this model or the DailyDialog dataset, please cite:
+
+```bibtex
+@inproceedings{li2017dailydialog,
+  title={DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset},
+  author={Li, Yanran and others},
+  booktitle={Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)},
+  year={2017}
+}
+
 ## Info
 License: Mit