| --- |
| license: apache-2.0 |
| datasets: |
| - facebook/empathetic_dialogues |
| language: |
| - id |
| metrics: |
| - bleu |
| - rouge |
| - accuracy |
| - f1 |
| base_model: |
| - muchad/idt5-base |
| tags: |
| - idmt |
| --- |
| # Indonesian Multitask Text Generation and Emotion Classification |
| This model provides a new refresher in the field of emotion-aware dialogue systems in Indonesian by creating the Indonesian Empathetic Dialogue Dataset and conducting multitask text generation and emotion classification training using pretrained idT5 |
|
|
| ## Model Details |
| ### Model Description |
| - **Developed by:** Adhitia Erfina, Tran Thi Oanh, Le-Hong Phuong |
| - **Funded by:** xxxxxxxxxxxxxxxxxxx |
| - **Model type:** Multitask Text Generation and Emotion Classification |
| - **Language(s) (NLP):** Indonesia |
| - **Finetuned from model:** muchad/idt5-base |
|
|
| ### Model Sources |
| - **Repository:** https://github.com/adhitia17/Multitask-Generative-Dialogue-and-Emotion-Classification-with-Indonesian-Empathetic-Dialogue-Dataset |
| - **Paper:** xxxxxxxxxxxxxxxxxxx |
|
|
| ## Uses |
| This model is designed for multitask text-to-text generation in Indonesian, specifically trained for: |
| 1. Dialogue Response Generation: Given a user utterance prefixed with dialog:, the model generates a relevant conversational response. |
| 2. Emotion Classification: Given a text prefixed with emosi:, the model predicts the underlying emotion expressed in the text. |
| 3. Context Understanding/Summarization (if applicable based on your training data): Given a text prefixed with konteks:, the model can perform tasks related to understanding or summarizing the provided context. |
| It's intended to be used directly via the transformers library in Python for applications requiring these specific capabilities in Indonesian. |
| ### Direct Use |
| ``` |
| import torch |
| from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
| |
| repo_id = "adhitia17/idmt" |
| |
| print(f"Loading tokenizer and model from {repo_id}...") |
| tokenizer = AutoTokenizer.from_pretrained(repo_id) |
| model = AutoModelForSeq2SeqLM.from_pretrained(repo_id) |
| |
| device = torch.device("cuda" if torch.cuda.is_available() else "cpu") |
| model.to(device) |
| print(f"Model loaded to device: {device}") |
| |
| def generate_response(input_text, task_prefix): |
| """Generates a response from the model for a given task.""" |
| full_input = f"{task_prefix}: {input_text}" |
| print(f"\nInput ({task_prefix}): {full_input}") |
| |
| input_ids = tokenizer(full_input, return_tensors="pt").input_ids.to(device) |
| |
| outputs = model.generate( |
| input_ids, |
| max_length=256, |
| num_beams=5, |
| early_stopping=True |
| ) |
| |
| decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| print(f"Output: {decoded_output}") |
| return decoded_output |
| |
| |
| print("\nInference examples complete.") |
| ``` |
|
|
|
|
| ## How to Get Started with the Model |
| Use the code below to get started with the model. |
| [More Information Needed] |
|
|
|
|
| ## Training Details |
| ### Training Data |
| Translated facebook/empathetic_dialogues to Indonesian (81.005 rows) |
| ### Training Procedure |
| Multitask Text Generation and Emotion Classification using T5Base |
| #### Preprocessing |
| facebook/empathetic_dialogues translated to Indonesian using facebook/nllb-200-1.3B |
| #### Training Hyperparameters |
| - **Learning Rate:** 1e-5 |
| - **Weight Decay:** 0.01 |
| - **Token:** 512 |
| - **Batch:** 64 |
| - **Epochs:** 40 |
| - **Warm Up Steps :** 500 |
| - **Optimizer:** Adam |
| - **Evaluation Metrics :** BLEU + ROUGE (text generation) & Accuracy + F1 (emotion classification) |
|
|
|
|
| ## Evaluation |
| Translated facebook/empathetic_dialogues to Indonesian (12.044 rows) |
| ### Testing Data & Metrics |
| #### Testing Data |
| Translated facebook/empathetic_dialogues to Indonesian (10.945 rows) |
| ### Results |
| - **BLEU:** 0.1071 |
| - **ROUGE:** 0.2264 |
| - **Accuracy:** 0.7064 |
| - **F1:** 0.7049 |
|
|
|
|
| ## Technical Specifications |
| #### GPU |
| 1x NVIDIA H100 with 80 GB HBM2e memory, and FP8 Tensor Core 3.958 TFLOPS. |
| #### Training Hours |
| ±18 hours |
| ## Citation |
| xxxxxxxxxxxxxxxxxxx |
