colable commited on
Commit
0a3bf58
·
verified ·
1 Parent(s): 566488a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -1,3 +1,25 @@
1
  ---
2
  license: mit
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - ko
5
  ---
6
+
7
+ # open-llama-2-ko based model with modified DPO dataset
8
+
9
+ This is an Korean Model based on
10
+ * [beomi/open-llama-2-ko-7b]
11
+
12
+ Dataset is modified from
13
+ * [SJ-Donald/orca-dpo-pairs-ko]
14
+
15
+ Parameters
16
+ ```
17
+ learning_rate: float = 3e-4
18
+ lr_scheduler: str = "cosine"
19
+ warmup_ratio: float = 0.1
20
+ lora_r: int = 16
21
+ lora_alpha: int = 16
22
+ lora_dropout: float = 0.05
23
+ optim='paged_adamw_32bit'
24
+ bf16=True
25
+ ```