---
license: apache-2.0
datasets:
- facebook/empathetic_dialogues
language:
- id
metrics:
- bleu
- rouge
- accuracy
- f1
base_model:
- muchad/idt5-base
tags:
- idmt
---
# Indonesian Multitask Text Generation and Emotion Classification
This model provides a new refresher in the field of emotion-aware dialogue systems in Indonesian by creating the Indonesian Empathetic Dialogue Dataset and conducting multitask text generation and emotion classification training using pretrained idT5

## Model Details
### Model Description
- **Developed by:** Adhitia Erfina, Tran Thi Oanh, Le-Hong Phuong
- **Funded by:** xxxxxxxxxxxxxxxxxxx
- **Model type:** Multitask Text Generation and Emotion Classification
- **Language(s) (NLP):** Indonesia
- **Finetuned from model:** muchad/idt5-base

### Model Sources 
- **Repository:** https://github.com/adhitia17/Multitask-Generative-Dialogue-and-Emotion-Classification-with-Indonesian-Empathetic-Dialogue-Dataset
- **Paper:** xxxxxxxxxxxxxxxxxxx

## Uses
This model is designed for multitask text-to-text generation in Indonesian, specifically trained for:
1. Dialogue Response Generation: Given a user utterance prefixed with dialog:, the model generates a relevant conversational response.
2. Emotion Classification: Given a text prefixed with emosi:, the model predicts the underlying emotion expressed in the text.
3. Context Understanding/Summarization (if applicable based on your training data): Given a text prefixed with konteks:, the model can perform tasks related to understanding or summarizing the provided context.
It's intended to be used directly via the transformers library in Python for applications requiring these specific capabilities in Indonesian.
### Direct Use
```
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

repo_id = "adhitia17/idmt"

print(f"Loading tokenizer and model from {repo_id}...")
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
print(f"Model loaded to device: {device}")

def generate_response(input_text, task_prefix):
    """Generates a response from the model for a given task."""
    full_input = f"{task_prefix}: {input_text}"
    print(f"\nInput ({task_prefix}): {full_input}")
    
    input_ids = tokenizer(full_input, return_tensors="pt").input_ids.to(device)
    
    outputs = model.generate(
        input_ids,
        max_length=256, 
        num_beams=5,
        early_stopping=True
    )
    
    decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print(f"Output: {decoded_output}")
    return decoded_output


print("\nInference examples complete.")
```


## How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]


## Training Details
### Training Data
Translated facebook/empathetic_dialogues to Indonesian (81.005 rows)
### Training Procedure
Multitask Text Generation and Emotion Classification using T5Base
#### Preprocessing
facebook/empathetic_dialogues translated to Indonesian using facebook/nllb-200-1.3B
#### Training Hyperparameters
- **Learning Rate:** 1e-5 
- **Weight Decay:** 0.01 
- **Token:** 512
- **Batch:** 64
- **Epochs:** 40
- **Warm Up Steps :** 500
- **Optimizer:** Adam
- **Evaluation Metrics :** BLEU + ROUGE (text generation) & Accuracy + F1 (emotion classification) 


## Evaluation
Translated facebook/empathetic_dialogues to Indonesian (12.044 rows)
### Testing Data & Metrics
#### Testing Data
Translated facebook/empathetic_dialogues to Indonesian (10.945 rows)
### Results
- **BLEU:** 0.1071
- **ROUGE:** 0.2264
- **Accuracy:** 0.7064
- **F1:** 0.7049


## Technical Specifications
#### GPU
1x NVIDIA H100 with 80 GB HBM2e memory, and FP8 Tensor Core 3.958 TFLOPS.
#### Training Hours
±18 hours
## Citation
xxxxxxxxxxxxxxxxxxx