Update README.md
Browse files
README.md
CHANGED
|
@@ -24,9 +24,9 @@ Unlike response-generation models, this version is trained to **predict client f
|
|
| 24 |
|
| 25 |
## π‘ Overview
|
| 26 |
|
| 27 |
-
- β
|
| 28 |
-
- β
|
| 29 |
-
- β
|
| 30 |
- β
Base Model: [`tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3`](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3)
|
| 31 |
|
| 32 |
---
|
|
@@ -95,17 +95,15 @@ Fine-tuning was performed using **QLoRA** with the following configuration:
|
|
| 95 |
|
| 96 |
### Dataset Split
|
| 97 |
|
| 98 |
-
- **Training
|
| 99 |
-
- **Train/Validation Split**: 90% train, 10% validation
|
| 100 |
|
| 101 |
### Hyperparameter Settings
|
| 102 |
|
| 103 |
-
- **Optimizer**: `
|
| 104 |
- **Warm-up Steps**: `100`
|
| 105 |
-
- **Learning Rate**: `
|
| 106 |
-
- **Epochs**: `
|
| 107 |
-
- **Batch Size**: `
|
| 108 |
-
- **Validation Frequency**: every 400 steps
|
| 109 |
|
| 110 |
---
|
| 111 |
|
|
@@ -131,5 +129,6 @@ If you use this model or dataset, please cite the following paper:
|
|
| 131 |
- [KokoroChat on GitHub (UEC-InabaLab)](https://github.com/UEC-InabaLab/KokoroChat)
|
| 132 |
- π€ **Model Variants**:
|
| 133 |
- [Llama-3.1-KokoroChat-Low](https://huggingface.co/UEC-InabaLab/Llama-3.1-KokoroChat-Low): fine-tuned on **3,870 dialogues** with client feedback scores **< 70**
|
|
|
|
| 134 |
- [Llama-3.1-KokoroChat-Full](https://huggingface.co/UEC-InabaLab/Llama-3.1-KokoroChat-Full): fine-tuned on **6,471 dialogues** with client feedback scores **β€ 98**
|
| 135 |
- π **Paper**: [ACL 2025 Paper](https://aclanthology.org/2025.acl-long.608/)
|
|
|
|
| 24 |
|
| 25 |
## π‘ Overview
|
| 26 |
|
| 27 |
+
- β
Task: Predict the **overall counseling quality score** as rated by the client
|
| 28 |
+
- β
Dataset: 6,589 dialogues with feedback scores between 0 and 100
|
| 29 |
+
- β
Data source: Text-based role-play by trained counselors
|
| 30 |
- β
Base Model: [`tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3`](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3)
|
| 31 |
|
| 32 |
---
|
|
|
|
| 95 |
|
| 96 |
### Dataset Split
|
| 97 |
|
| 98 |
+
- **Training/Validation/Test ratio**: 8:1:1
|
|
|
|
| 99 |
|
| 100 |
### Hyperparameter Settings
|
| 101 |
|
| 102 |
+
- **Optimizer**: `adamw_torch_fused`
|
| 103 |
- **Warm-up Steps**: `100`
|
| 104 |
+
- **Learning Rate**: `2e-4`
|
| 105 |
+
- **Epochs**: `4`
|
| 106 |
+
- **Batch Size**: `4`
|
|
|
|
| 107 |
|
| 108 |
---
|
| 109 |
|
|
|
|
| 129 |
- [KokoroChat on GitHub (UEC-InabaLab)](https://github.com/UEC-InabaLab/KokoroChat)
|
| 130 |
- π€ **Model Variants**:
|
| 131 |
- [Llama-3.1-KokoroChat-Low](https://huggingface.co/UEC-InabaLab/Llama-3.1-KokoroChat-Low): fine-tuned on **3,870 dialogues** with client feedback scores **< 70**
|
| 132 |
+
- [Llama-3.1-KokoroChat-High](https://huggingface.co/UEC-InabaLab/Llama-3.1-KokoroChat-High): fine-tuned on **2,601 dialogues** with client feedback scores between **70 and 98**
|
| 133 |
- [Llama-3.1-KokoroChat-Full](https://huggingface.co/UEC-InabaLab/Llama-3.1-KokoroChat-Full): fine-tuned on **6,471 dialogues** with client feedback scores **β€ 98**
|
| 134 |
- π **Paper**: [ACL 2025 Paper](https://aclanthology.org/2025.acl-long.608/)
|