mintaeng commited on
Commit
b76f15f
ยท
verified ยท
1 Parent(s): fa5b373

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -49,8 +49,9 @@ LoRA์™€ SFT Trainer ๋ฐฉ์‹์„ ์‚ฌ์šฉํ•˜์˜€์Šต๋‹ˆ๋‹ค.
49
 
50
  #### Training Hyperparameters
51
 
52
- - **Training regime:** bf16 mixed precision
53
- - r=32,
 
54
  lora_alpha=64, # QLoRA : alpha = r/2 // LoRA : alpha =r*2
55
  lora_dropout=0.05,
56
  target_modules=[
@@ -62,6 +63,7 @@ LoRA์™€ SFT Trainer ๋ฐฉ์‹์„ ์‚ฌ์šฉํ•˜์˜€์Šต๋‹ˆ๋‹ค.
62
  "up_proj",
63
  "down_proj",
64
  ], # ํƒ€๊ฒŸ ๋ชจ๋“ˆ
 
65
 
66
  ### Result
67
  https://github.com/lucide99/Chatbot_FutFut
 
49
 
50
  #### Training Hyperparameters
51
 
52
+ - **Training regime:** bf16 mixed precision
53
+ - ```
54
+ r=32,
55
  lora_alpha=64, # QLoRA : alpha = r/2 // LoRA : alpha =r*2
56
  lora_dropout=0.05,
57
  target_modules=[
 
63
  "up_proj",
64
  "down_proj",
65
  ], # ํƒ€๊ฒŸ ๋ชจ๋“ˆ
66
+ ```
67
 
68
  ### Result
69
  https://github.com/lucide99/Chatbot_FutFut