|
|
--- |
|
|
language: |
|
|
- ko |
|
|
--- |
|
|
## Model description |
|
|
|
|
|
Intent |
|
|
- 의도없음 |
|
|
- 조명전원켜짐 |
|
|
- 조명전원꺼짐 |
|
|
- 조명밝게설정 |
|
|
- 조명어둡게설정 |
|
|
- 조명색상설정 |
|
|
- 조명밝기조절 |
|
|
- 조명색상모드설정 |
|
|
- 조명차갑게설정 |
|
|
- 조명따뜻하게설정 |
|
|
- 모드실행 |
|
|
- 상태조회 |
|
|
- 조명전체켜짐 |
|
|
- 조명전체꺼짐 |
|
|
- 조명켜짐예약등록 |
|
|
- 조명꺼짐예약등록 |
|
|
- 조명예약취소 |
|
|
- 별칭등록 |
|
|
|
|
|
Slots |
|
|
- O |
|
|
- mode |
|
|
- time |
|
|
- home |
|
|
- device |
|
|
- nickname |
|
|
- percent |
|
|
- color_red |
|
|
- color_yellow |
|
|
- color_blue |
|
|
- color_mode |
|
|
|
|
|
|
|
|
## Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- train_batch_size: 32 |
|
|
- eval_batch_size: 32 |
|
|
- gradient_accumulation_steps: 8 |
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=2.0e-05 |
|
|
- lr_scheduler_type: cosine |
|
|
- lr_scheduler_warmup_ratio: 0.1 |
|
|
- num_epochs: 10 |
|
|
- weight_decay: 0.01 |
|
|
- intent_loss_weight: 0.6 |
|
|
|
|
|
## Evaluation results |
|
|
|
|
|
```plain |
|
|
Intent report: |
|
|
label precision recall f1 support |
|
|
의도없음 (label_id: 0) 83.33 100.00 90.91 5 |
|
|
조명전원켜짐 (label_id: 1) 100.00 100.00 100.00 10 |
|
|
조명전원꺼짐 (label_id: 2) 100.00 100.00 100.00 10 |
|
|
조명밝게설정 (label_id: 3) 76.92 100.00 86.96 10 |
|
|
조명어둡게설정 (label_id: 4) 100.00 100.00 100.00 10 |
|
|
조명색상설정 (label_id: 5) 83.33 100.00 90.91 10 |
|
|
조명밝기조절 (label_id: 6) 100.00 100.00 100.00 10 |
|
|
조명색상모드설정 (label_id: 7) 100.00 100.00 100.00 10 |
|
|
조명차갑게설정 (label_id: 8) 90.00 100.00 94.74 9 |
|
|
조명따뜻하게설정 (label_id: 9) 100.00 90.91 95.24 11 |
|
|
모드실행 (label_id: 10) 100.00 100.00 100.00 10 |
|
|
상태조회 (label_id: 11) 100.00 80.00 88.89 10 |
|
|
조명전체켜짐 (label_id: 12) 100.00 100.00 100.00 10 |
|
|
조명전체꺼짐 (label_id: 13) 100.00 100.00 100.00 10 |
|
|
조명켜짐예약등록 (label_id: 14) 100.00 100.00 100.00 10 |
|
|
조명꺼짐예약등록 (label_id: 15) 90.91 100.00 95.24 10 |
|
|
조명예약취소 (label_id: 16) 100.00 90.00 94.74 10 |
|
|
별칭등록 (label_id: 17) 100.00 60.00 75.00 10 |
|
|
------------------- |
|
|
micro avg 95.43 95.43 95.43 175 |
|
|
macro avg 95.81 95.61 95.15 175 |
|
|
weighted avg 96.22 95.43 95.27 175 |
|
|
Slot report: |
|
|
label precision recall f1 support |
|
|
O (label_id: 0) 95.16 89.29 92.13 308 |
|
|
mode (label_id: 1) 90.91 100.00 95.24 10 |
|
|
time (label_id: 2) 95.74 100.00 97.83 45 |
|
|
home (label_id: 3) 91.85 96.12 93.94 129 |
|
|
device (label_id: 4) 84.62 92.26 88.27 155 |
|
|
nickname (label_id: 5) 100.00 42.86 60.00 7 |
|
|
percent (label_id: 6) 100.00 100.00 100.00 10 |
|
|
color_red (label_id: 7) 80.00 100.00 88.89 4 |
|
|
color_yellow (label_id: 8) 100.00 100.00 100.00 3 |
|
|
color_blue (label_id: 9) 100.00 100.00 100.00 2 |
|
|
color_mode (label_id: 10) 100.00 90.00 94.74 10 |
|
|
------------------- |
|
|
micro avg 91.95 91.95 91.95 683 |
|
|
macro avg 94.39 91.87 91.91 683 |
|
|
weighted avg 92.25 91.95 91.88 683 |
|
|
``` |
