Improve language tag
#1
by
lbourdois
- opened
README.md
CHANGED
|
@@ -1,131 +1,143 @@
|
|
| 1 |
-
---
|
| 2 |
-
tags:
|
| 3 |
-
- finance
|
| 4 |
-
- accounting
|
| 5 |
-
- stock
|
| 6 |
-
- quant
|
| 7 |
-
- economics
|
| 8 |
-
language:
|
| 9 |
-
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
-
|
| 13 |
-
|
| 14 |
-
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
|
| 75 |
-
|
| 76 |
-
|
|
| 77 |
-
|
|
| 78 |
-
|
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
|
| 92 |
-
|
| 93 |
-
|
|
| 94 |
-
|
|
| 95 |
-
|
|
| 96 |
-
|
|
| 97 |
-
|
|
| 98 |
-
|
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
|
| 111 |
-
|
| 112 |
-
|
|
| 113 |
-
|
|
| 114 |
-
|
|
| 115 |
-
|
|
| 116 |
-
|
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 131 |
```
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- finance
|
| 4 |
+
- accounting
|
| 5 |
+
- stock
|
| 6 |
+
- quant
|
| 7 |
+
- economics
|
| 8 |
+
language:
|
| 9 |
+
- zho
|
| 10 |
+
- eng
|
| 11 |
+
- fra
|
| 12 |
+
- spa
|
| 13 |
+
- por
|
| 14 |
+
- deu
|
| 15 |
+
- ita
|
| 16 |
+
- rus
|
| 17 |
+
- jpn
|
| 18 |
+
- kor
|
| 19 |
+
- vie
|
| 20 |
+
- tha
|
| 21 |
+
- ara
|
| 22 |
+
license: apache-2.0
|
| 23 |
+
datasets:
|
| 24 |
+
- aiqwe/FinShibainu
|
| 25 |
+
base_model:
|
| 26 |
+
- Qwen/Qwen2.5-7B-Instruct
|
| 27 |
+
pipeline_tag: question-answering
|
| 28 |
+
library_name: transformers
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
# FinShibainu Model Card
|
| 32 |
+
|
| 33 |
+
+ github: [https://github.com/aiqwe/FinShibainu](https://github.com/aiqwe/FinShibainu)
|
| 34 |
+
+ dataset: [https://huggingface.co/datasets/aiqwe/FinShibainu](https://huggingface.co/datasets/aiqwe/FinShibainu)
|
| 35 |
+
|
| 36 |
+
๋ชจ๋ธ์ [KRX LLM ๊ฒฝ์ง๋ํ ๋ฆฌ๋๋ณด๋](https://krxbench.koscom.co.kr/)์์ ์ฐ์์์ ์์ํ shibainu24 ๋ชจ๋ธ์
๋๋ค. ๋ชจ๋ธ์ ๊ธ์ต, ํ๊ณ ๋ฑ ๊ธ์ต๊ด๋ จ ์ง์์ ๋ํ Text Generation์ ์ ๊ณตํฉ๋๋ค.
|
| 37 |
+
|
| 38 |
+
+ Vanilla model : [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
|
| 39 |
+
|
| 40 |
+
๋ฐ์ดํฐ์
์์ง ๋ฐ ํ์ต์ ๊ด๋ จ๋ ์ฝ๋๋ [https://github.com/aiqwe/FinShibainu](https://github.com/aiqwe/FinShibainu)์ ์์ธํ๊ฒ ๊ณต๊ฐ๋์ด ์์ต๋๋ค.
|
| 41 |
+
|
| 42 |
+
# Usage
|
| 43 |
+
[https://github.com/aiqwe/FinShibainu](https://github.com/aiqwe/FinShibainu)์ example์ ์ฐธ์กฐํ๋ฉด ์ฝ๊ฒ inference๋ฅผ ํด๋ณผ ์ ์์ต๋๋ค.
|
| 44 |
+
๋๋ถ๋ถ์ Inference๋ RTX-3090 ์ด์์์ ๋จ์ผ GPU ๊ฐ๋ฅํฉ๋๋ค.
|
| 45 |
+
|
| 46 |
+
```shell
|
| 47 |
+
pip install vllm
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
import pandas as pd
|
| 52 |
+
from vllm import LLM
|
| 53 |
+
|
| 54 |
+
inputs = [
|
| 55 |
+
"์ธํ์์ฅ์์ ์ผ๋ณธ ์ํ์ ๋ฏธ๊ตญ ๋ฌ๋ฌ์ ํ์จ์ด ๋ ์์ฅ์์ ์ฝ๊ฐ์ ์ฐจ์ด๋ฅผ ๋ณด์ด๊ณ ์๋ค. ์ด๋ ๋ฌด์ํ ์ด์ต์ ์ป๊ธฐ ์ํ ์ ์ ํ ๊ฑฐ๋ ์ ๋ต์ ๋ฌด์์ธ๊ฐ?",
|
| 56 |
+
"์ ์ฃผ์ธ์๊ถ๋ถ์ฌ์ฑ(BW)์์ ์ฑ๊ถ์๊ฐ ์ ์ฃผ์ธ์๊ถ์ ํ์ฌํ์ง ์์ ๊ฒฝ์ฐ ์ด๋ค ์ผ์ด ๋ฐ์ํ๋๊ฐ?",
|
| 57 |
+
"๊ณต๋งค๋(Short Selling)์ ๋ํ ์ค๋ช
์ผ๋ก ์ณ์ง ์์ ๊ฒ์ ๋ฌด์์
๋๊น?"
|
| 58 |
+
]
|
| 59 |
+
|
| 60 |
+
llm = LLM(model="aiqwe/krx-llm-competition", tensor_parallel_size=1)
|
| 61 |
+
sampling_params = SamplingParams(temperature=0.7, max_tokens=128)
|
| 62 |
+
outputs = llm.generate(inputs, sampling_params)
|
| 63 |
+
for o in outputs:
|
| 64 |
+
print(o.prompt)
|
| 65 |
+
print(o.outputs[0].text)
|
| 66 |
+
print("*"*100)
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
# Model Card
|
| 70 |
+
| Contents | Spec |
|
| 71 |
+
|--------------------------------|-------------------------------------|
|
| 72 |
+
| Base model | Qwen2.5-7B-Instruct |
|
| 73 |
+
| dtype | bfloat16 |
|
| 74 |
+
| PEFT | LoRA (r=8, alpha=64) |
|
| 75 |
+
| Learning Rate | 1e-5 (varies by further training) |
|
| 76 |
+
| LRScheduler | Cosine (warm-up: 0.05%) |
|
| 77 |
+
| Optimizer | AdamW |
|
| 78 |
+
| Distributed / Efficient Tuning | DeepSpeed v3, Flash Attention |
|
| 79 |
+
|
| 80 |
+
# Datset Card
|
| 81 |
+
Reference ๋ฐ์ดํฐ์
์ ์ผ๋ถ ์ ์๊ถ ๊ด๊ณ๋ก ์ธํด Link๋ก ์ ๊ณตํฉ๋๋ค.
|
| 82 |
+
MCQA์ QA ๋ฐ์ดํฐ์
์ [https://huggingface.co/datasets/aiqwe/FinShibainu](https://huggingface.co/datasets/aiqwe/FinShibainu)์ผ๋ก ๊ณต๊ฐํฉ๋๋ค.
|
| 83 |
+
๋ํ [https://github.com/aiqwe/FinShibainu](https://github.com/aiqwe/FinShibainu)๋ฅผ ์ด์ฉํ๋ฉด ๋ค์ํ ์ ํธ๋ฆฌํฐ ๊ธฐ๋ฅ์ ์ ๊ณตํ๋ฉฐ, ๋ฐ์ดํฐ ์์ฑ Pipeline์ ์ฐธ์กฐํ ์ ์์ต๋๋ค.
|
| 84 |
+
|
| 85 |
+
## References
|
| 86 |
+
| ๋ฐ์ดํฐ๋ช
| url |
|
| 87 |
+
|-----------------------------------|------------------------------------------------------------------------------------------|
|
| 88 |
+
| ํ๊ตญ์ํ ๊ฒฝ์ ๊ธ์ต ์ฉ์ด 700์ | [Link](https://www.bok.or.kr/portal/bbs/B0000249/view.do?nttId=235017&menuNo=200765) |
|
| 89 |
+
| ์ฌ๋ฌดํ๊ณ ํฉ์ฑ ๋ฐ์ดํฐ | ์์ฒด ์ ์ |
|
| 90 |
+
| ๊ธ์ต๊ฐ๋
์ฉ์ด์ฌ์ | [Link](https://terms.naver.com/list.naver?cid=42088&categoryId=42088) |
|
| 91 |
+
| web-text.synthetic.dataset-50k | [Link](https://huggingface.co/datasets/Cartinoe5930/web_text_synthetic_dataset_50k) |
|
| 92 |
+
| ์ง์๊ฒฝ์ ์ฉ์ด์ฌ์ | [Link](https://terms.naver.com/list.naver?cid=43668&categoryId=43668) |
|
| 93 |
+
| ํ๊ตญ๊ฑฐ๋์ ๋น์ ๊ธฐ ๊ฐํ๋ฌผ | [Link](http://open.krx.co.kr/contents/OPN04/04020000/OPN04020000.jsp#b8943a5f87282cde0d653d1ae73431c9=1) |
|
| 94 |
+
| ํ๊ตญ๊ฑฐ๋์๊ท์ | [Link](https://law.krx.co.kr/las/TopFrame.jsp&KRX) |
|
| 95 |
+
| ์ด๋ณดํฌ์์ ์ฆ๊ถ๋ฐ๋ผ์ก๊ธฐ | [Link](https://main.krxverse.co.kr/_contents/ACA/02010200/file/220104_beginner.pdf) |
|
| 96 |
+
| ์ฒญ์๋
์ ์ํ ์ฆ๊ถํฌ์ | [Link](https://main.krxverse.co.kr/_contents/ACA/02010200/file/220104_teen.pdf) |
|
| 97 |
+
| ๊ธฐ์
์ฌ์
๋ณด๊ณ ์ ๊ณต์์๋ฃ | [Link](https://opendart.fss.or.kr/) |
|
| 98 |
+
| ์์ฌ๊ฒฝ์ ์ฉ์ด์ฌ์ | [Link](https://terms.naver.com/list.naver?cid=43668&categoryId=43668) |
|
| 99 |
+
|
| 100 |
+
## MCQA
|
| 101 |
+
MCQA ๋ฐ์ดํฐ๋ Reference๋ฅผ ๊ธฐ๋ฐ์ผ๋ก ๋ค์ง์ ๋คํ ๋ฌธ์ ๋ฅผ ์์ฑํ ๋ฐ์ดํฐ์
์
๋๋ค. ๋ฌธ์ ์ ๋ต ๋ฟ๋ง ์๋๋ผ Reasoning ํ
์คํธ๊น์ง ์์ฑํ์ฌ ํ์ต์ ์ถ๊ฐํ์์ต๋๋ค.
|
| 102 |
+
ํ์ต์ ์ฌ์ฉ๋ ๋ฐ์ดํฐ๋ ์ฝ 4.5๋ง๊ฐ ๋ฐ์ดํฐ์
์ด๋ฉฐ, tiktoken์ o200k_base(gpt-4o, gpt-4o-mini Tokenizer)๋ฅผ ๊ธฐ์ค์ผ๋ก ์ด 2์ฒ๋ง๊ฐ์ ํ ํฐ์ผ๋ก ํ์ต๋์์ต๋๋ค.
|
| 103 |
+
| ๋ฐ์ดํฐ๋ช
| ๋ฐ์ดํฐ ์ | ํ ํฐ ์ |
|
| 104 |
+
|--------------------------------------|-----------|--------------|
|
| 105 |
+
| ํ๊ตญ์ํ ๊ฒฝ์ ๊ธ์ต ์ฉ์ด 700์ | 1,203 | 277,114 |
|
| 106 |
+
| ์ฌ๋ฌดํ๊ณ ๋ชฉ์ฐจ๋ฅผ ์ด์ฉํ ํฉ์ฑ๋ฐ์ดํฐ | 451 | 99,770 |
|
| 107 |
+
| ๊ธ์ต๊ฐ๋
์ฉ์ด์ฌ์ | 827 | 214,297 |
|
| 108 |
+
| hf_web_text_synthetic_dataset_50k | 25,461 | 7,563,529 |
|
| 109 |
+
| ์ง์๊ฒฝ์ ์ฉ์ด์ฌ์ | 2,314 | 589,763 |
|
| 110 |
+
| ํ๊ตญ๊ฑฐ๋์ ๋น์ ๊ธฐ ๊ฐํ๋ฌผ | 1,183 | 230,148 |
|
| 111 |
+
| ํ๊ตญ๊ฑฐ๋์๊ท์ | 3,015 | 580,556 |
|
| 112 |
+
| ์ด๋ณดํฌ์์ ์ฆ๊ถ๋ฐ๋ผ์ก๊ธฐ | 599 | 116,472 |
|
| 113 |
+
| ์ฒญ์๋
์ ์ํ ์ฆ๊ถ ํฌ์ | 408 | 77,037 |
|
| 114 |
+
| ๊ธฐ์
์ฌ์
๋ณด๊ณ ์ ๊ณต์์๋ฃ | 3,574 | 629,807 |
|
| 115 |
+
| ์์ฌ๊ฒฝ์ ์ฉ์ด์ฌ์ | 7,410 | 1,545,842 |
|
| 116 |
+
| **ํฉ๊ณ** | **46,445**| **19,998,931**|
|
| 117 |
+
|
| 118 |
+
## QA
|
| 119 |
+
QA ๋ฐ์ดํฐ๋ Reference์ ์ง๋ฌธ์ ํจ๊ป Input์ผ๋ก ๋ฐ์ ์์ฑํ ๋ต๋ณ๊ณผ Reference ์์ด ์ง๋ฌธ๋ง์ Input์ผ๋ก ๋ฐ์ ์์ฑํ ๋ต๋ณ 2๊ฐ์ง๋ก ๊ตฌ์ฑ๋ฉ๋๋ค.
|
| 120 |
+
Reference๋ฅผ ์ ๊ณต๋ฐ์ผ๋ฉด ๋ชจ๋ธ์ ๋ณด๋ค ์ ํํ ๋ต๋ณ์ ํ์ง๋ง ๋ชจ๋ธ๋ง์ ์ง์์ด ์ ํ๋์ด ๋ต๋ณ์ด ์ข๋ ์งง์์ง๊ฑฐ๋ ๋ค์์ฑ์ด ์ค์ด๋ค๊ฒ ๋ฉ๋๋ค.
|
| 121 |
+
์ด 4.8๋ง๊ฐ์ ๋ฐ์ดํฐ์
๊ณผ 2์ต๊ฐ์ ํ ํฐ์ผ๋ก ํ์ต๋์์ต๋๋ค.
|
| 122 |
+
| ๋ฐ์ดํฐ๋ช
| ๋ฐ์ดํฐ ์ | ํ ํฐ ์ |
|
| 123 |
+
|--------------------------------------|-----------|--------------|
|
| 124 |
+
| ํ๊ตญ์ํ ๊ฒฝ์ ๊ธ์ต ์ฉ์ด 700์ | 1,023 | 846,970 |
|
| 125 |
+
| ๊ธ์ต๊ฐ๋
์ฉ์ด์ฌ์ | 4,128 | 3,181,831 |
|
| 126 |
+
| ์ง์๊ฒฝ์ ์ฉ์ด์ฌ์ | 6,526 | 5,311,890 |
|
| 127 |
+
| ํ๊ตญ๊ฑฐ๋์ ๋น์ ๊ธฐ ๊ฐํ๋ฌผ | 1,510 | 1,089,342 |
|
| 128 |
+
| ํ๊ตญ๊ฑฐ๋์๊ท์ | 4,858 | 3,587,059 |
|
| 129 |
+
| ๊ธฐ์
์ฌ์
๋ณด๊ณ ์ ๊ณต์์๋ฃ | 3,574 | 629,807 |
|
| 130 |
+
| ์์ฌ๊ฒฝ์ ์ฉ์ด์ฌ์ | 29,920 | 5,981,839 |
|
| 131 |
+
| **ํฉ๊ณ** | **47,965**| **199,998,931**|
|
| 132 |
+
|
| 133 |
+
# Citation
|
| 134 |
+
```bibitex
|
| 135 |
+
@misc{jaylee2024finshibainu,
|
| 136 |
+
author = {Jay Lee},
|
| 137 |
+
title = {FinShibainu: Korean specified finance model},
|
| 138 |
+
year = {2024},
|
| 139 |
+
publisher = {GitHub},
|
| 140 |
+
journal = {GitHub repository},
|
| 141 |
+
url = {https://github.com/aiqwe/FinShibainu}
|
| 142 |
+
}
|
| 143 |
```
|