Update README.md
Browse files
README.md
CHANGED
|
@@ -20,6 +20,7 @@ tags:
|
|
| 20 |
|
| 21 |
(โป ์ด ๋ชจ๋ธ์ AI Hub์ [์ด๊ฑฐ๋ AI ํฌ์ค์ผ์ด ์ง์ ์๋ต ๋ฐ์ดํฐ](https://aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&dataSetSn=71762)๋ก ํ์ตํ ๋ชจ๋ธ์
๋๋ค.)
|
| 22 |
|
|
|
|
| 23 |
## 2. Model
|
| 24 |
|
| 25 |
**(1) Self Alignment Pretraining (SAP)**
|
|
@@ -38,7 +39,6 @@ Multi Similarity Loss๋ฅผ ์ด์ฉํด **๋์ผํ ์ฝ๋์ ์ฉ์ด** ๊ฐ์ ๋์
|
|
| 38 |
- SapBERT-KO-EN : [https://huggingface.co/snumin44/sap-bert-ko-en](https://huggingface.co/snumin44/sap-bert-ko-en)
|
| 39 |
- Github : [https://github.com/snumin44/SapBERT-KO-EN](https://github.com/snumin44/SapBERT-KO-EN)
|
| 40 |
|
| 41 |
-
|
| 42 |
**(2) Dense Passage Retrieval (DPR)**
|
| 43 |
|
| 44 |
SapBERT-KO-EN์ ๊ฒ์ ๋ชจ๋ธ๋ก ๋ง๋ค๊ธฐ ์ํด ์ถ๊ฐ์ ์ธ Fine-tuning์ ํด์ผ ํฉ๋๋ค.
|
|
@@ -53,6 +53,7 @@ Bi-Encoder ๊ตฌ์กฐ๋ก ์ง์์ ํ
์คํธ์ ์ ์ฌ๋๋ฅผ ๊ณ์ฐํ๋ DPR ๋ฐฉ์
|
|
| 53 |
|
| 54 |
- Github : [https://github.com/snumin44/DPR-KO](https://github.com/snumin44/DPR-KO)
|
| 55 |
|
|
|
|
| 56 |
## 3. Training
|
| 57 |
|
| 58 |
**(1) Self Alignment Pretraining (SAP)**
|
|
@@ -76,10 +77,24 @@ SapBERT-KO-EN ํ์ต์ ํ์ฉํ ๋ฒ ์ด์ค ๋ชจ๋ธ ๋ฐ ํ์ดํผ ํ๋ผ๋ฏธํฐ๋
|
|
| 76 |
|
| 77 |
Fine-tuning์ ํ์ฉํ ๋ฒ ์ด์ค ๋ชจ๋ธ ๋ฐ ํ์ดํผ ํ๋ผ๋ฏธํฐ๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
|
| 78 |
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
|
| 82 |
## 4. Example
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
|
|
|
| 20 |
|
| 21 |
(โป ์ด ๋ชจ๋ธ์ AI Hub์ [์ด๊ฑฐ๋ AI ํฌ์ค์ผ์ด ์ง์ ์๋ต ๋ฐ์ดํฐ](https://aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&dataSetSn=71762)๋ก ํ์ตํ ๋ชจ๋ธ์
๋๋ค.)
|
| 22 |
|
| 23 |
+
|
| 24 |
## 2. Model
|
| 25 |
|
| 26 |
**(1) Self Alignment Pretraining (SAP)**
|
|
|
|
| 39 |
- SapBERT-KO-EN : [https://huggingface.co/snumin44/sap-bert-ko-en](https://huggingface.co/snumin44/sap-bert-ko-en)
|
| 40 |
- Github : [https://github.com/snumin44/SapBERT-KO-EN](https://github.com/snumin44/SapBERT-KO-EN)
|
| 41 |
|
|
|
|
| 42 |
**(2) Dense Passage Retrieval (DPR)**
|
| 43 |
|
| 44 |
SapBERT-KO-EN์ ๊ฒ์ ๋ชจ๋ธ๋ก ๋ง๋ค๊ธฐ ์ํด ์ถ๊ฐ์ ์ธ Fine-tuning์ ํด์ผ ํฉ๋๋ค.
|
|
|
|
| 53 |
|
| 54 |
- Github : [https://github.com/snumin44/DPR-KO](https://github.com/snumin44/DPR-KO)
|
| 55 |
|
| 56 |
+
|
| 57 |
## 3. Training
|
| 58 |
|
| 59 |
**(1) Self Alignment Pretraining (SAP)**
|
|
|
|
| 77 |
|
| 78 |
Fine-tuning์ ํ์ฉํ ๋ฒ ์ด์ค ๋ชจ๋ธ ๋ฐ ํ์ดํผ ํ๋ผ๋ฏธํฐ๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
|
| 79 |
|
| 80 |
+
- Model : SapBERT-KO-EN(klue/bert-base)
|
| 81 |
+
- Dataset : **์ด๊ฑฐ๋ AI ํฌ์ค์ผ์ด ์ง์ ์๋ต ๋ฐ์ดํฐ(AI Hub)**
|
| 82 |
+
- Epochs : 10
|
| 83 |
+
- Batch Size : 64
|
| 84 |
+
- Dropout : 0.1
|
| 85 |
+
- Pooler : 'cls'
|
| 86 |
|
| 87 |
|
| 88 |
## 4. Example
|
| 89 |
+
์ด ๋ชจ๋ธ์ Question์ ์ธ์ฝ๋ฉํ๋ ๋ชจ๋ธ๋ก, Context ๋ชจ๋ธ๊ณผ ํจ๊ป ์ฌ์ฉํด์ผ ํฉ๋๋ค.
|
| 90 |
+
๋์ผํ ์ง๋ณ์ ๊ดํ ์ง๋ฌธ๊ณผ ํ
์คํธ๊ฐ ๋์ ์ ์ฌ๋๋ฅผ ๋ณด์ธ๋ค๋ ์ฌ์ค์ ํ์ธํ ์ ์์ต๋๋ค.
|
| 91 |
+
|
| 92 |
+
```python
|
| 93 |
+
```
|
| 94 |
|
| 95 |
|
| 96 |
+
## Citing
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
```
|
| 100 |
|