Update README.md
Browse files
README.md
CHANGED
|
@@ -4,10 +4,8 @@ tags: []
|
|
| 4 |
---
|
| 5 |
|
| 6 |
# HowRU-KoELECTRA-Emotion-Classifier
|
| 7 |
-
## Model Details
|
| 8 |
-
|
| 9 |
-
### Model Description
|
| 10 |
|
|
|
|
| 11 |
์ด ๋ชจ๋ธ์ ํ๊ตญ์ด ์ผ์ ํ
์คํธ(ํนํ ์ผ๊ธฐ/์ฌ๋ฆฌ ๊ธฐ๋ก)์ ๋ํ๋๋ ๊ฐ์ ์ ์๋์ผ๋ก ๋ถ๋ฅํ๊ธฐ ์ํด ๊ตฌ์ถ๋ ๊ฐ์ ๋ถ๋ฅ ๋ชจ๋ธ์
๋๋ค.
|
| 12 |
๊ธฐ๋ฐ ๋ชจ๋ธ์ monologg/koelectra-base-v3-discriminator์ด๋ฉฐ, ๊ฐ์ ๋ถ์ Task์ ์ต์ ํ๋๋๋ก 8๊ฐ ๊ฐ์ ๋ผ๋ฒจ(๊ธฐ์จ, ์ค๋ , ํ๋ฒํจ, ๋๋ผ์, ๋ถ์พํจ, ๋๋ ค์, ์ฌํ, ๋ถ๋
ธ) ๊ตฌ์กฐ๋ก ํ์ธํ๋๋์์ต๋๋ค.
|
| 13 |
|
|
@@ -16,8 +14,7 @@ tags: []
|
|
| 16 |
- **License:** MIT
|
| 17 |
- **Finetuned from model:** monologg/koelectra-base-v3-discriminator
|
| 18 |
|
| 19 |
-
|
| 20 |
-
|
| 21 |
์ด ๋ชจ๋ธ์ ์
๋ ฅ๋ ํ๊ตญ์ด ๋ฌธ์ฅ ๋๋ ์ผ๊ธฐ ํ
์คํธ์ ์ฃผ์ ๊ฐ์ ์ ์๋ 8๊ฐ ํด๋์ค ์ค ํ๋๋ก ๋ถ๋ฅํฉ๋๋ค.
|
| 22 |
| Emotion (๊ฐ์ ) |
|
| 23 |
|----------------|
|
|
@@ -33,7 +30,6 @@ tags: []
|
|
| 33 |
---
|
| 34 |
|
| 35 |
## How to Get Started with the Model
|
| 36 |
-
|
| 37 |
```python
|
| 38 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 39 |
import torch.nn.functional as F
|
|
@@ -60,7 +56,6 @@ print(predict("์ค๋ ํ๋ฃจ๊ฐ ์ ๋ํ ํ๋ค๊ณ ์ง์ณค๋ค."))
|
|
| 60 |
## Training Details
|
| 61 |
|
| 62 |
### Training Data
|
| 63 |
-
|
| 64 |
1. [LimYeri/kor-diary-emotion_v2]("https://huggingface.co/datasets/LimYeri/kor-diary-emotion_v2")
|
| 65 |
2. [qowlsdud/CounselGPT]("https://huggingface.co/datasets/qowlsdud/CounselGPT")
|
| 66 |
|
|
@@ -69,14 +64,12 @@ print(predict("์ค๋ ํ๋ฃจ๊ฐ ์ ๋ํ ํ๋ค๊ณ ์ง์ณค๋ค."))
|
|
| 69 |
- **Validation:** 10,000ํ
|
| 70 |
|
| 71 |
### Training Procedure
|
| 72 |
-
|
| 73 |
- **Base Model**: [monologg/koelectra-base-v3-discriminator]("https://huggingface.co/monologg/koelectra-base-v3-discriminator")
|
| 74 |
- **Objective**: Single-label classification
|
| 75 |
- **Precision**: fp16 mixed precision
|
| 76 |
- **Max Length**: 512
|
| 77 |
|
| 78 |
-
|
| 79 |
-
|
| 80 |
- **num_train_epochs**: 3
|
| 81 |
- **learning_rate**: 3e-5
|
| 82 |
- **weight_decay**: 0.02
|
|
@@ -88,7 +81,6 @@ print(predict("์ค๋ ํ๋ฃจ๊ฐ ์ ๋ํ ํ๋ค๊ณ ์ง์ณค๋ค."))
|
|
| 88 |
---
|
| 89 |
|
| 90 |
## Performance
|
| 91 |
-
|
| 92 |
| Metric | Score |
|
| 93 |
|-----------------|--------|
|
| 94 |
| **Eval Accuracy** | 0.95 |
|
|
@@ -99,7 +91,6 @@ print(predict("์ค๋ ํ๋ฃจ๊ฐ ์ ๋ํ ํ๋ค๊ณ ์ง์ณค๋ค."))
|
|
| 99 |
## Model Architecture
|
| 100 |
|
| 101 |
### 1) ELECTRA Encoder (Base-size)
|
| 102 |
-
|
| 103 |
- **Hidden size:** 768
|
| 104 |
- **Layers:** 12 Transformer blocks
|
| 105 |
- **Attention heads:** 12
|
|
@@ -108,7 +99,6 @@ print(predict("์ค๋ ํ๋ฃจ๊ฐ ์ ๋ํ ํ๋ค๊ณ ์ง์ณค๋ค."))
|
|
| 108 |
- **Dropout:** 0.1
|
| 109 |
|
| 110 |
### 2) Classification Head
|
| 111 |
-
|
| 112 |
๊ฐ์ 8๊ฐ ํด๋์ค๋ฅผ ์์ธกํ๊ธฐ ์ํ ์ถ๊ฐ ๋ถ๋ฅ ํค๋:
|
| 113 |
|
| 114 |
- **Dense Layer**: 768 โ 768
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
# HowRU-KoELECTRA-Emotion-Classifier
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
+
## Model Description
|
| 9 |
์ด ๋ชจ๋ธ์ ํ๊ตญ์ด ์ผ์ ํ
์คํธ(ํนํ ์ผ๊ธฐ/์ฌ๋ฆฌ ๊ธฐ๋ก)์ ๋ํ๋๋ ๊ฐ์ ์ ์๋์ผ๋ก ๋ถ๋ฅํ๊ธฐ ์ํด ๊ตฌ์ถ๋ ๊ฐ์ ๋ถ๋ฅ ๋ชจ๋ธ์
๋๋ค.
|
| 10 |
๊ธฐ๋ฐ ๋ชจ๋ธ์ monologg/koelectra-base-v3-discriminator์ด๋ฉฐ, ๊ฐ์ ๋ถ์ Task์ ์ต์ ํ๋๋๋ก 8๊ฐ ๊ฐ์ ๋ผ๋ฒจ(๊ธฐ์จ, ์ค๋ , ํ๋ฒํจ, ๋๋ผ์, ๋ถ์พํจ, ๋๋ ค์, ์ฌํ, ๋ถ๋
ธ) ๊ตฌ์กฐ๋ก ํ์ธํ๋๋์์ต๋๋ค.
|
| 11 |
|
|
|
|
| 14 |
- **License:** MIT
|
| 15 |
- **Finetuned from model:** monologg/koelectra-base-v3-discriminator
|
| 16 |
|
| 17 |
+
## Emotion Classes
|
|
|
|
| 18 |
์ด ๋ชจ๋ธ์ ์
๋ ฅ๋ ํ๊ตญ์ด ๋ฌธ์ฅ ๋๋ ์ผ๊ธฐ ํ
์คํธ์ ์ฃผ์ ๊ฐ์ ์ ์๋ 8๊ฐ ํด๋์ค ์ค ํ๋๋ก ๋ถ๋ฅํฉ๋๋ค.
|
| 19 |
| Emotion (๊ฐ์ ) |
|
| 20 |
|----------------|
|
|
|
|
| 30 |
---
|
| 31 |
|
| 32 |
## How to Get Started with the Model
|
|
|
|
| 33 |
```python
|
| 34 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 35 |
import torch.nn.functional as F
|
|
|
|
| 56 |
## Training Details
|
| 57 |
|
| 58 |
### Training Data
|
|
|
|
| 59 |
1. [LimYeri/kor-diary-emotion_v2]("https://huggingface.co/datasets/LimYeri/kor-diary-emotion_v2")
|
| 60 |
2. [qowlsdud/CounselGPT]("https://huggingface.co/datasets/qowlsdud/CounselGPT")
|
| 61 |
|
|
|
|
| 64 |
- **Validation:** 10,000ํ
|
| 65 |
|
| 66 |
### Training Procedure
|
|
|
|
| 67 |
- **Base Model**: [monologg/koelectra-base-v3-discriminator]("https://huggingface.co/monologg/koelectra-base-v3-discriminator")
|
| 68 |
- **Objective**: Single-label classification
|
| 69 |
- **Precision**: fp16 mixed precision
|
| 70 |
- **Max Length**: 512
|
| 71 |
|
| 72 |
+
### Training Hyperparameters
|
|
|
|
| 73 |
- **num_train_epochs**: 3
|
| 74 |
- **learning_rate**: 3e-5
|
| 75 |
- **weight_decay**: 0.02
|
|
|
|
| 81 |
---
|
| 82 |
|
| 83 |
## Performance
|
|
|
|
| 84 |
| Metric | Score |
|
| 85 |
|-----------------|--------|
|
| 86 |
| **Eval Accuracy** | 0.95 |
|
|
|
|
| 91 |
## Model Architecture
|
| 92 |
|
| 93 |
### 1) ELECTRA Encoder (Base-size)
|
|
|
|
| 94 |
- **Hidden size:** 768
|
| 95 |
- **Layers:** 12 Transformer blocks
|
| 96 |
- **Attention heads:** 12
|
|
|
|
| 99 |
- **Dropout:** 0.1
|
| 100 |
|
| 101 |
### 2) Classification Head
|
|
|
|
| 102 |
๊ฐ์ 8๊ฐ ํด๋์ค๋ฅผ ์์ธกํ๊ธฐ ์ํ ์ถ๊ฐ ๋ถ๋ฅ ํค๋:
|
| 103 |
|
| 104 |
- **Dense Layer**: 768 โ 768
|