Update README.md
Browse files
README.md
CHANGED
|
@@ -10,40 +10,72 @@ tags:
|
|
| 10 |
- 4bit
|
| 11 |
---
|
| 12 |
|
| 13 |
-
#
|
| 14 |
|
| 15 |
-
`taetae030/fin-term-model`์
|
| 16 |
-
Meta์
|
| 17 |
-
ํ์ต์๋ ์๋์ instruct-style ๋ฐ์ดํฐ์
์ ์ฌ์ฉํ์ต๋๋ค:
|
| 18 |
|
| 19 |
-
|
|
|
|
| 20 |
|
| 21 |
---
|
| 22 |
|
| 23 |
-
##
|
| 24 |
|
| 25 |
-
| ํญ๋ชฉ |
|
| 26 |
-
|
| 27 |
-
| ๊ธฐ๋ฐ ๋ชจ๋ธ | Meta-LLaMA-3-8B-Instruct
|
| 28 |
-
|
|
| 29 |
-
|
|
| 30 |
-
|
|
| 31 |
-
|
|
| 32 |
-
|
|
| 33 |
-
|
|
| 34 |
-
| ๋ผ์ด์ ์ค | Apache-2.0 |
|
| 35 |
|
| 36 |
---
|
| 37 |
|
| 38 |
-
##
|
| 39 |
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
|
|
|
| 43 |
|
| 44 |
---
|
| 45 |
|
| 46 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
```python
|
| 49 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -51,7 +83,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
| 51 |
model = AutoModelForCausalLM.from_pretrained("taetae030/fin-term-model")
|
| 52 |
tokenizer = AutoTokenizer.from_pretrained("taetae030/fin-term-model")
|
| 53 |
|
| 54 |
-
prompt = "
|
| 55 |
inputs = tokenizer(prompt, return_tensors="pt")
|
| 56 |
-
outputs = model.generate(**inputs, max_new_tokens=
|
| 57 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
|
|
| 10 |
- 4bit
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# ๐ฌ KoFinGPT: ํ๊ตญ์ด ๊ธ์ต ์ฉ์ด ์ค๋ช
ํนํ LLM (LoRA + LLaMA-3)
|
| 14 |
|
| 15 |
+
`taetae030/fin-term-model`์ **ํ๊ตญ์ด ๊ธ์ต ์ฉ์ด๋ฅผ ์ฝ๊ฒ ์ค๋ช
ํด์ฃผ๋ ์ฑ๋ด ํนํ LLM**์
๋๋ค.
|
| 16 |
+
Meta์ ์ต์ LLaMA-3 8B Instruct ๋ชจ๋ธ์ ๊ธฐ๋ฐ์ผ๋ก, ํ๊ตญ์ํ ๋ฑ ๊ณต๊ณต๊ธฐ๊ด์ ๊ธ์ต ๋ฌธ์๋ฅผ ํ์ฉํ **์ง๋ฌธ-์๋ต instruct ๋ฐ์ดํฐ์
**์ผ๋ก ํ๋๋์์ต๋๋ค.
|
|
|
|
| 17 |
|
| 18 |
+
์ด ๋ชจ๋ธ์ ๊ธ์ต ์ด๋ณด์๋ ์ดํดํ ์ ์๋๋ก **์ ๋ฌธ ์ฉ์ด๋ฅผ ๊ฐ๊ฒฐํ๊ฒ ์ค๋ช
**ํ๋ ๋ฐ ์ด์ ์ ๋ง์ท์ผ๋ฉฐ,
|
| 19 |
+
LoRA ๋ฐฉ์์ผ๋ก ๊ฒฝ๋ํ๋ ํ์ธํ๋ ๊ตฌ์กฐ๋ฅผ ์ฌ์ฉํฉ๋๋ค.
|
| 20 |
|
| 21 |
---
|
| 22 |
|
| 23 |
+
## ๐ง ๋ชจ๋ธ ๊ฐ์
|
| 24 |
|
| 25 |
+
| ํญ๋ชฉ | ์ค๋ช
|
|
| 26 |
+
|-----------------|----------------------------------------------------------------------|
|
| 27 |
+
| ๊ธฐ๋ฐ ๋ชจ๋ธ | Meta-LLaMA-3-8B-Instruct |
|
| 28 |
+
| ํ๋ ๋ฐฉ์ | LoRA (Low-Rank Adaptation) + Supervised Fine-Tuning (SFT) |
|
| 29 |
+
| ์์ํ | 4bit (bnb.nf4, bitsandbytes ์ฌ์ฉ) |
|
| 30 |
+
| ์ฌ์ฉ ์ธ์ด | ํ๊ตญ์ด |
|
| 31 |
+
| ์ฃผ์ ์ฉ๋ | ๊ธ์ต ์ฉ์ด ์ค๋ช
, ์ง์์๋ต ๊ธฐ๋ฐ ์ฑ๋ด |
|
| 32 |
+
| ํ์ต ๋ฐ์ดํฐ | [fin-term-instruct](https://huggingface.co/datasets/taetae030/fin-term-instruct) |
|
| 33 |
+
| ๋ผ์ด์ ์ค | Apache-2.0 |
|
|
|
|
| 34 |
|
| 35 |
---
|
| 36 |
|
| 37 |
+
## โจ ๋ชจ๋ธ ํน์ง
|
| 38 |
|
| 39 |
+
- ๐ก ๊ธ์ต ์ฉ์ด์ ๋ํ ์์ฐ์ด ์ง์์๋ต ์ง์
|
| 40 |
+
- ๐ ๊ณต๊ณต๊ธฐ๊ด ๋ฌธ์๋ฅผ ๊ธฐ๋ฐ์ผ๋ก ๋์ ์ ๋ขฐ์ฑ
|
| 41 |
+
- ๐งฉ LoRA ๊ตฌ์กฐ ๋ณํฉ ๊ฐ๋ฅ (`merge_and_unload()` ์ฌ์ฉ ๊ฐ๋ฅ)
|
| 42 |
+
- ๐ฌ ํ๊ตญ์ด ๊ธฐ๋ฐ ์ฑ๋ด ํ๊ฒฝ์ ์ ํฉ
|
| 43 |
|
| 44 |
---
|
| 45 |
|
| 46 |
+
## ๐ฆ ํ์ต ๋ฐ์ดํฐ ์ถ์ฒ
|
| 47 |
+
|
| 48 |
+
๋ชจ๋ธ ํ์ต์ ์ฌ์ฉ๋ instruct ๋ฐ์ดํฐ์
์ **AI ํ๋ธ์ ๊ณต๊ณต ๋ฐ์ดํฐ**๋ฅผ ๊ธฐ๋ฐ์ผ๋ก ๊ตฌ์ถ๋์์ต๋๋ค.
|
| 49 |
+
|
| 50 |
+
- ๐ **AI ํ๋ธ โ ๊ธ์ตยท๋ฒ๋ฅ ๋ฌธ์ ๊ธฐ๊ณ๋
ํด ๋ฐ์ดํฐ**
|
| 51 |
+
๐ [๋งํฌ ๋ฐ๋ก๊ฐ๊ธฐ](https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=&topMenu=&aihubDataSe=data&dataSetSn=71610)
|
| 52 |
+
|
| 53 |
+
- **๋ฐ์ดํฐ ๊ตฌ์ถ๋
๋**: 2022๋
|
| 54 |
+
- **์ ์ฒด ๊ตฌ์ถ๋**: ์ฝ 40๋ง ๊ฑด
|
| 55 |
+
- **์ ๋ณ ๊ธฐ์ค**: ์ ์ฒด ๋ฐ์ดํฐ ์ค `๊ธ์ต๊ฒฝ์ ` ๋ถ์ผ(์ฝ 17.3%)๋ง ์ ๋ณ ์ฌ์ฉ
|
| 56 |
+
- **ํ์**: JSON (์ง๋ฌธ - ์ง๋ฌธ - ๋ต๋ณ ๊ตฌ์ฑ)
|
| 57 |
+
|
| 58 |
+
- **์ถ์ฒ ๊ธฐ๊ด**:
|
| 59 |
+
- ํ๊ตญ์ํ
|
| 60 |
+
- ๊ธ์ต์์ํ
|
| 61 |
+
- ๊ธ์ต๊ฐ๋
์
|
| 62 |
+
- ๊ตญํ์
๋ฒ์กฐ์ฌ์ฒ
|
| 63 |
+
- ๋ฒ์ ์ฒ
|
| 64 |
+
- ํ๊ตญ๊ธ์ต์ฐ๊ตฌ์ ๋ฑ
|
| 65 |
+
|
| 66 |
+
- โ๏ธ ํด๋น ์๋ณธ์ ๊ธฐ๋ฐ์ผ๋ก GPT ๋ชจ๋ธ์ ํ์ฉํด ๊ฐ๊ฒฐํ๊ณ ์ผ๊ด๋ instruct ํฌ๋งท(`instruction`, `input`, `output`)์ผ๋ก ์ฌ์ ์ ํ์์ต๋๋ค.
|
| 67 |
+
|
| 68 |
+
---
|
| 69 |
+
|
| 70 |
+
## ๐งช ์์
|
| 71 |
+
|
| 72 |
+
**์
๋ ฅ**: `๊ธฐ์ถํตํ๋ ๋ฌด์์ธ๊ฐ์?`
|
| 73 |
+
**๋ชจ๋ธ ์๋ต**:
|
| 74 |
+
> ๊ธฐ์ถํตํ๋ ๊ตญ์ ๊ฑฐ๋์์ ๋๋ฆฌ ์ฌ์ฉ๋๋ ๊ธฐ์ค์ด ๋๋ ํตํ๋ฅผ ์๋ฏธํฉ๋๋ค. ํ์ฌ๋ ๋ฏธ๊ตญ ๋ฌ๋ฌ๊ฐ ๋ํ์ ์ธ ๊ธฐ์ถํตํ์
๋๋ค.
|
| 75 |
+
|
| 76 |
+
---
|
| 77 |
+
|
| 78 |
+
## ๐ง ์ฌ์ฉ๋ฒ (Inference ์์ )
|
| 79 |
|
| 80 |
```python
|
| 81 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 83 |
model = AutoModelForCausalLM.from_pretrained("taetae030/fin-term-model")
|
| 84 |
tokenizer = AutoTokenizer.from_pretrained("taetae030/fin-term-model")
|
| 85 |
|
| 86 |
+
prompt = "ํตํ์ค์ํ๋?"
|
| 87 |
inputs = tokenizer(prompt, return_tensors="pt")
|
| 88 |
+
outputs = model.generate(**inputs, max_new_tokens=128)
|
| 89 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|