Update README.md
Browse files
README.md
CHANGED
|
@@ -41,17 +41,17 @@ Command r plus ๋ชจ๋ธ์ ์ด์ฉํ์ฌ ์์ฒด ๊ตฌ์ถํ RAG ํนํ ๋ฐ์ดํฐ์
,
|
|
| 41 |
```
|
| 42 |
|
| 43 |
## ํ์ต ํ๊ฒฝ ๋ฐ ํ๋ผ๋ฏธํฐ
|
| 44 |
-
ํ๋ ํ๊ฒฝ : H100(80GB) * 8
|
| 45 |
-
-tokenizer_model_mex_length 4500
|
| 46 |
-
-use_flash_attn True
|
| 47 |
-
-num_train_epochs 3.0
|
| 48 |
-
-weight_decay 0.001
|
| 49 |
-
-lr_scheduler_type "linear"
|
| 50 |
-
-per_device_train_batch_size 1
|
| 51 |
-
-gradient_accumulation_steps 64
|
| 52 |
-
-learning_rate 5e-06
|
| 53 |
-
-bf16 True
|
| 54 |
-
-deepspeed ds_stage2.json
|
| 55 |
|
| 56 |
## ์ฌ์ฉ ๋ฐ์ดํฐ์
|
| 57 |
- AIhub 16 ํ์ ๋ฌธ์ ๋์ ๊ธฐ๊ณ๋
ํด ๋ฐ์ดํฐ
|
|
|
|
| 41 |
```
|
| 42 |
|
| 43 |
## ํ์ต ํ๊ฒฝ ๋ฐ ํ๋ผ๋ฏธํฐ
|
| 44 |
+
- ํ๋ ํ๊ฒฝ : H100(80GB) * 8
|
| 45 |
+
- tokenizer_model_mex_length 4500
|
| 46 |
+
- use_flash_attn True
|
| 47 |
+
- num_train_epochs 3.0
|
| 48 |
+
- weight_decay 0.001
|
| 49 |
+
- lr_scheduler_type "linear"
|
| 50 |
+
- per_device_train_batch_size 1
|
| 51 |
+
- gradient_accumulation_steps 64
|
| 52 |
+
- learning_rate 5e-06
|
| 53 |
+
- bf16 True
|
| 54 |
+
- deepspeed ds_stage2.json
|
| 55 |
|
| 56 |
## ์ฌ์ฉ ๋ฐ์ดํฐ์
|
| 57 |
- AIhub 16 ํ์ ๋ฌธ์ ๋์ ๊ธฐ๊ณ๋
ํด ๋ฐ์ดํฐ
|