taetae030 commited on
Commit
47e980d
ยท
verified ยท
1 Parent(s): fb19ac0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -23
README.md CHANGED
@@ -10,40 +10,72 @@ tags:
10
  - 4bit
11
  ---
12
 
13
- # ๐Ÿงพ KoFinGPT: ํ•œ๊ตญ์–ด ๊ธˆ์œต ์šฉ์–ด ์„ค๋ช… ์ฑ—๋ด‡ ๋ชจ๋ธ (LoRA + LLaMA-3 ๊ธฐ๋ฐ˜)
14
 
15
- `taetae030/fin-term-model`์€ ํ•œ๊ตญ์–ด ๊ธˆ์œต ๋ถ„์•ผ ์งˆ์˜์‘๋‹ต์— ํŠนํ™”๋œ LLM์ž…๋‹ˆ๋‹ค.
16
- Meta์˜ **LLaMA-3 8B Instruct**๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ, ํ•œ๊ตญ์€ํ–‰ ๊ด€๋ จ ๊ธˆ์œต ์šฉ์–ด๋ฅผ ์‰ฝ๊ฒŒ ์„ค๋ช…ํ•ด์ฃผ๋Š” ์ฑ—๋ด‡์„ ๋งŒ๋“ค๊ธฐ ์œ„ํ•ด ํŠœ๋‹๋˜์—ˆ์Šต๋‹ˆ๋‹ค.
17
- ํ•™์Šต์—๋Š” ์•„๋ž˜์˜ instruct-style ๋ฐ์ดํ„ฐ์…‹์„ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค:
18
 
19
- ๐Ÿ‘‰ [๐Ÿ’พ `taetae030/fin-term-instruct`](https://huggingface.co/datasets/taetae030/fin-term-instruct)
 
20
 
21
  ---
22
 
23
- ## ๐Ÿ› ๏ธ ๋ชจ๋ธ ์„ธ๋ถ€ ์ •๋ณด
24
 
25
- | ํ•ญ๋ชฉ | ๋‚ด์šฉ |
26
- |-----------------|----------------------------------------------|
27
- | ๊ธฐ๋ฐ˜ ๋ชจ๋ธ | Meta-LLaMA-3-8B-Instruct |
28
- | ํ•™์Šต ๋ฐฉ์‹ | LoRA (Low-Rank Adaptation) |
29
- | ํŠœ๋‹ ๋ฐฉ์‹ | Supervised Fine-Tuning (SFT) |
30
- | ์–‘์žํ™” | 4bit (bnb.nf4, bitsandbytes) |
31
- | ์–ธ์–ด | ํ•œ๊ตญ์–ด |
32
- | ์ฃผ์š” ์šฉ๋„ | ๊ธˆ์œต ์šฉ์–ด ์งˆ์˜์‘๋‹ต, ์„ค๋ช… ์ฑ—๋ด‡ |
33
- | ๋ฐ์ดํ„ฐ์…‹ | [`fin-term-instruct`](https://huggingface.co/datasets/taetae030/fin-term-instruct) |
34
- | ๋ผ์ด์„ ์Šค | Apache-2.0 |
35
 
36
  ---
37
 
38
- ## ๐Ÿงช ์‚ฌ์šฉ ์˜ˆ์‹œ
39
 
40
- **์งˆ๋ฌธ**: ํ†ตํ™”์ •์ฑ…์ด๋ž€?
41
- **์‘๋‹ต**:
42
- > ํ†ตํ™”์ •์ฑ…์€ ์ค‘์•™์€ํ–‰์ด ๊ธˆ๋ฆฌ์™€ ํ†ตํ™”๋Ÿ‰์„ ์กฐ์ ˆํ•˜์—ฌ ๊ฒฝ์ œ ์•ˆ์ •์„ ์ถ”๊ตฌํ•˜๋Š” ์ •์ฑ…์ž…๋‹ˆ๋‹ค.
 
43
 
44
  ---
45
 
46
- ## ๐Ÿ’ป ์‚ฌ์šฉ๋ฒ•
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ```python
49
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -51,7 +83,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
51
  model = AutoModelForCausalLM.from_pretrained("taetae030/fin-term-model")
52
  tokenizer = AutoTokenizer.from_pretrained("taetae030/fin-term-model")
53
 
54
- prompt = "ํ†ตํ™”์ •์ฑ…์ด๋ž€?"
55
  inputs = tokenizer(prompt, return_tensors="pt")
56
- outputs = model.generate(**inputs, max_new_tokens=100)
57
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 
10
  - 4bit
11
  ---
12
 
13
+ # ๐Ÿ’ฌ KoFinGPT: ํ•œ๊ตญ์–ด ๊ธˆ์œต ์šฉ์–ด ์„ค๋ช… ํŠนํ™” LLM (LoRA + LLaMA-3)
14
 
15
+ `taetae030/fin-term-model`์€ **ํ•œ๊ตญ์–ด ๊ธˆ์œต ์šฉ์–ด๋ฅผ ์‰ฝ๊ฒŒ ์„ค๋ช…ํ•ด์ฃผ๋Š” ์ฑ—๋ด‡ ํŠนํ™” LLM**์ž…๋‹ˆ๋‹ค.
16
+ Meta์˜ ์ตœ์‹  LLaMA-3 8B Instruct ๋ชจ๋ธ์„ ๊ธฐ๋ฐ˜์œผ๋กœ, ํ•œ๊ตญ์€ํ–‰ ๋“ฑ ๊ณต๊ณต๊ธฐ๊ด€์˜ ๊ธˆ์œต ๋ฌธ์„œ๋ฅผ ํ™œ์šฉํ•œ **์งˆ๋ฌธ-์‘๋‹ต instruct ๋ฐ์ดํ„ฐ์…‹**์œผ๋กœ ํŠœ๋‹๋˜์—ˆ์Šต๋‹ˆ๋‹ค.
 
17
 
18
+ ์ด ๋ชจ๋ธ์€ ๊ธˆ์œต ์ดˆ๋ณด์ž๋„ ์ดํ•ดํ•  ์ˆ˜ ์žˆ๋„๋ก **์ „๋ฌธ ์šฉ์–ด๋ฅผ ๊ฐ„๊ฒฐํ•˜๊ฒŒ ์„ค๋ช…**ํ•˜๋Š” ๋ฐ ์ดˆ์ ์„ ๋งž์ท„์œผ๋ฉฐ,
19
+ LoRA ๋ฐฉ์‹์œผ๋กœ ๊ฒฝ๋Ÿ‰ํ™”๋œ ํŒŒ์ธํŠœ๋‹ ๊ตฌ์กฐ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.
20
 
21
  ---
22
 
23
+ ## ๐Ÿง  ๋ชจ๋ธ ๊ฐœ์š”
24
 
25
+ | ํ•ญ๋ชฉ | ์„ค๋ช… |
26
+ |-----------------|----------------------------------------------------------------------|
27
+ | ๊ธฐ๋ฐ˜ ๋ชจ๋ธ | Meta-LLaMA-3-8B-Instruct |
28
+ | ํŠœ๋‹ ๋ฐฉ์‹ | LoRA (Low-Rank Adaptation) + Supervised Fine-Tuning (SFT) |
29
+ | ์–‘์žํ™” | 4bit (bnb.nf4, bitsandbytes ์‚ฌ์šฉ) |
30
+ | ์‚ฌ์šฉ ์–ธ์–ด | ํ•œ๊ตญ์–ด |
31
+ | ์ฃผ์š” ์šฉ๋„ | ๊ธˆ์œต ์šฉ์–ด ์„ค๋ช…, ์งˆ์˜์‘๋‹ต ๊ธฐ๋ฐ˜ ์ฑ—๋ด‡ |
32
+ | ํ•™์Šต ๋ฐ์ดํ„ฐ | [fin-term-instruct](https://huggingface.co/datasets/taetae030/fin-term-instruct) |
33
+ | ๋ผ์ด์„ ์Šค | Apache-2.0 |
 
34
 
35
  ---
36
 
37
+ ## โœจ ๋ชจ๋ธ ํŠน์ง•
38
 
39
+ - ๐Ÿ’ก ๊ธˆ์œต ์šฉ์–ด์— ๋Œ€ํ•œ ์ž์—ฐ์–ด ์งˆ์˜์‘๋‹ต ์ง€์›
40
+ - ๐Ÿ“„ ๊ณต๊ณต๊ธฐ๊ด€ ๋ฌธ์„œ๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ๋†’์€ ์‹ ๋ขฐ์„ฑ
41
+ - ๐Ÿงฉ LoRA ๊ตฌ์กฐ ๋ณ‘ํ•ฉ ๊ฐ€๋Šฅ (`merge_and_unload()` ์‚ฌ์šฉ ๊ฐ€๋Šฅ)
42
+ - ๐Ÿ’ฌ ํ•œ๊ตญ์–ด ๊ธฐ๋ฐ˜ ์ฑ—๋ด‡ ํ™˜๊ฒฝ์— ์ ํ•ฉ
43
 
44
  ---
45
 
46
+ ## ๐Ÿ“ฆ ํ•™์Šต ๋ฐ์ดํ„ฐ ์ถœ์ฒ˜
47
+
48
+ ๋ชจ๋ธ ํ•™์Šต์— ์‚ฌ์šฉ๋œ instruct ๋ฐ์ดํ„ฐ์…‹์€ **AI ํ—ˆ๋ธŒ์˜ ๊ณต๊ณต ๋ฐ์ดํ„ฐ**๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ๊ตฌ์ถ•๋˜์—ˆ์Šต๋‹ˆ๋‹ค.
49
+
50
+ - ๐Ÿ“‚ **AI ํ—ˆ๋ธŒ โ€“ ๊ธˆ์œตยท๋ฒ•๋ฅ  ๋ฌธ์„œ ๊ธฐ๊ณ„๋…ํ•ด ๋ฐ์ดํ„ฐ**
51
+ ๐Ÿ‘‰ [๋งํฌ ๋ฐ”๋กœ๊ฐ€๊ธฐ](https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=&topMenu=&aihubDataSe=data&dataSetSn=71610)
52
+
53
+ - **๋ฐ์ดํ„ฐ ๊ตฌ์ถ•๋…„๋„**: 2022๋…„
54
+ - **์ „์ฒด ๊ตฌ์ถ•๋Ÿ‰**: ์•ฝ 40๋งŒ ๊ฑด
55
+ - **์„ ๋ณ„ ๊ธฐ์ค€**: ์ „์ฒด ๋ฐ์ดํ„ฐ ์ค‘ `๊ธˆ์œต๊ฒฝ์ œ` ๋ถ„์•ผ(์•ฝ 17.3%)๋งŒ ์„ ๋ณ„ ์‚ฌ์šฉ
56
+ - **ํ˜•์‹**: JSON (์ง€๋ฌธ - ์งˆ๋ฌธ - ๋‹ต๋ณ€ ๊ตฌ์„ฑ)
57
+
58
+ - **์ถœ์ฒ˜ ๊ธฐ๊ด€**:
59
+ - ํ•œ๊ตญ์€ํ–‰
60
+ - ๊ธˆ์œต์œ„์›ํšŒ
61
+ - ๊ธˆ์œต๊ฐ๋…์›
62
+ - ๊ตญํšŒ์ž…๋ฒ•์กฐ์‚ฌ์ฒ˜
63
+ - ๋ฒ•์ œ์ฒ˜
64
+ - ํ•œ๊ตญ๊ธˆ์œต์—ฐ๊ตฌ์› ๋“ฑ
65
+
66
+ - โš™๏ธ ํ•ด๋‹น ์›๋ณธ์„ ๊ธฐ๋ฐ˜์œผ๋กœ GPT ๋ชจ๋ธ์„ ํ™œ์šฉํ•ด ๊ฐ„๊ฒฐํ•˜๊ณ  ์ผ๊ด€๋œ instruct ํฌ๋งท(`instruction`, `input`, `output`)์œผ๋กœ ์žฌ์ •์ œํ•˜์˜€์Šต๋‹ˆ๋‹ค.
67
+
68
+ ---
69
+
70
+ ## ๐Ÿงช ์˜ˆ์‹œ
71
+
72
+ **์ž…๋ ฅ**: `๊ธฐ์ถ•ํ†ตํ™”๋ž€ ๋ฌด์—‡์ธ๊ฐ€์š”?`
73
+ **๋ชจ๋ธ ์‘๋‹ต**:
74
+ > ๊ธฐ์ถ•ํ†ตํ™”๋Š” ๊ตญ์ œ ๊ฑฐ๋ž˜์—์„œ ๋„๋ฆฌ ์‚ฌ์šฉ๋˜๋Š” ๊ธฐ์ค€์ด ๋˜๋Š” ํ†ตํ™”๋ฅผ ์˜๋ฏธํ•ฉ๋‹ˆ๋‹ค. ํ˜„์žฌ๋Š” ๋ฏธ๊ตญ ๋‹ฌ๋Ÿฌ๊ฐ€ ๋Œ€ํ‘œ์ ์ธ ๊ธฐ์ถ•ํ†ตํ™”์ž…๋‹ˆ๋‹ค.
75
+
76
+ ---
77
+
78
+ ## ๐Ÿ”ง ์‚ฌ์šฉ๋ฒ• (Inference ์˜ˆ์ œ)
79
 
80
  ```python
81
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
83
  model = AutoModelForCausalLM.from_pretrained("taetae030/fin-term-model")
84
  tokenizer = AutoTokenizer.from_pretrained("taetae030/fin-term-model")
85
 
86
+ prompt = "ํ†ตํ™”์Šค์™€ํ”„๋ž€?"
87
  inputs = tokenizer(prompt, return_tensors="pt")
88
+ outputs = model.generate(**inputs, max_new_tokens=128)
89
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))