File size: 418 Bytes
4c8cc1d 0a3bf58 4c8cc1d 0a3bf58 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
---
license: mit
language:
- ko
---
# open-llama-2-ko based model with modified DPO dataset
This is an Korean Model based on
* [beomi/open-llama-2-ko-7b]
Dataset is modified from
* [SJ-Donald/orca-dpo-pairs-ko]
Parameters
```
learning_rate: float = 3e-4
lr_scheduler: str = "cosine"
warmup_ratio: float = 0.1
lora_r: int = 16
lora_alpha: int = 16
lora_dropout: float = 0.05
optim='paged_adamw_32bit'
bf16=True
``` |