sel303's picture
Update README.md
0c78ea3 verified
metadata
license: llama3

LLaMA3 8B Instruct - Fine-Tuned Model

์ด ๋ชจ๋ธ์€ LLaMA3 8B Instruct ๊ธฐ๋ฐ˜์œผ๋กœ AIHub ๋ฐ์ดํ„ฐ์…‹๊ณผ ์ž์ฒด ์ œ์ž‘ํ•œ ์ปค์Šคํ…€ ๋ฐ์ดํ„ฐ์…‹์„ ํ™œ์šฉํ•ด ํŒŒ์ธํŠœ๋‹ํ•œ ๋ฒ„์ „์ž…๋‹ˆ๋‹ค.
๋ฏผ๊ฐ ๋ฐ์ดํ„ฐ๋ฅผ ํฌํ•จํ•  ์ˆ˜ ์žˆ์œผ๋ฏ€๋กœ ์‚ฌ์šฉ ์‹œ ์ฃผ์˜๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.


๐Ÿ“˜ Model Details

  • Base Model: LLaMA3 8B Instruct
  • Trainable Parameters: ์ „์ฒด ํŒŒ๋ผ๋ฏธํ„ฐ์˜ ์•ฝ 10%๋งŒ ํ•™์Šต ๊ฐ€๋Šฅํ•˜๋„๋ก ์„ค์ • (LoRA, QLoRA ๋˜๋Š” ๊ธฐํƒ€ PEFT ๋ฐฉ์‹ ์‚ฌ์šฉ ๊ฐ€๋Šฅ)
  • Fine-tuning Data:
    • AIHub ๊ณต๊ฐœ ๋ฐ์ดํ„ฐ์…‹
    • ์ž์ฒด ์ˆ˜์ง‘ ๋ฐ ๊ตฌ์ถ•ํ•œ ๋„๋ฉ”์ธ ํŠนํ™” ๋ฐ์ดํ„ฐ

โš™๏ธ Generation Configuration

Parameter Value
max_new_tokens 1024
temperature 0.75
repetition_penalty 1.1 ~ 1.2
do_sample True
top_k 5

โš ๏ธ ์ฃผ์˜์‚ฌํ•ญ

  • ์ด ๋ชจ๋ธ์€ ํ•™์Šต ๋ฐ์ดํ„ฐ์— ๋ฏผ๊ฐ ์ •๋ณด๋ฅผ ํฌํ•จํ•˜๊ณ  ์žˆ์„ ๊ฐ€๋Šฅ์„ฑ์ด ์žˆ์œผ๋ฏ€๋กœ, ์‹ค์ œ ์„œ๋น„์Šค๋‚˜ ์‘๋‹ต ํ™œ์šฉ ์‹œ ๋ฐ์ดํ„ฐ ๋ณด์•ˆ ๋ฐ ํ”„๋ผ์ด๋ฒ„์‹œ ๋ณดํ˜ธ์— ์œ ์˜ํ•ด ์ฃผ์„ธ์š”.