pathcosmos commited on
Commit
82f5b3e
Β·
verified Β·
1 Parent(s): e68bfa3

Document detailed JANSORI evaluations and recommended inference settings

Browse files
Files changed (2) hide show
  1. README.md +84 -0
  2. SHA256SUMS +1 -1
README.md CHANGED
@@ -89,6 +89,90 @@ Once upon a time, in a quiet village
89
  , there lived a poor old woman. She was a widow, and she had no children. She was very old, and she was very poor. She had no money, and she had no food. She was very lonely, and she was very poor. She was very old, and she was very poor. She
90
  ```
91
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
92
  ## ν•™μŠ΅ 데이터
93
 
94
  ν•œκ΅­μ–΄ 쀑심 3B Dense Base λͺ¨λΈμ€ 47,000μŠ€ν…μ—μ„œ λˆ„μ  98,566,144,000토큰을 ν•™μŠ΅ν–ˆλ‹€. μ•„λž˜ μˆ˜μΉ˜λŠ” ν•™μŠ΅ μ‹œ 반볡 λ…ΈμΆœμ„ ν¬ν•¨ν•œ μΆœμ²˜λ³„ μž…λ ₯ 토큰 μˆ˜λ‹€.
 
89
  , there lived a poor old woman. She was a widow, and she had no children. She was very old, and she was very poor. She had no money, and she had no food. She was very lonely, and she was very poor. She was very old, and she was very poor. She
90
  ```
91
 
92
+ ## H200μ—μ„œ μΈ‘μ •ν•œ ν’ˆμ§ˆκ³Ό μ‹€ν–‰ μ„€μ •
93
+
94
+ 이 κ²°κ³ΌλŠ” 47,000μŠ€ν…μ˜ [JANSORI 배포 컀밋](https://huggingface.co/pathcosmos/JANSORI/tree/e68bfa3cb31e9589f4481a2e31d6d75c731fec9d)에 μ μš©ν•œλ‹€. BF16은 vLLM 0.11.2, Q8_0Β·Q4_K_M은 Ollama 0.40.16으둜 같은 물리 H200 ν•œ μž₯μ—μ„œ 각각 μ‹€ν–‰ν–ˆλ‹€. 이전 배포 κ²€μ¦μ˜ 6개 κ³ μ • μž…λ ₯은 ν˜•μ‹λ³„λ‘œ 6/6 μž¬ν˜„λλ‹€.
95
+
96
+ ### ν•œκ΅­μ–΄Β·μ˜μ–΄Β·μˆ˜ν•™Β·μ½”λ“œ
97
+
98
+ 자체 생성 μ‹œν—˜μ€ ν˜•μ‹λ‹Ή 98개 μž…λ ₯이닀. 짧은 사싀 질문 24개, 색상 독해 16개, μ‚°μˆ˜ 32개, ν•¨μˆ˜ 12개, 이야기 이어쓰기 8개, νšŒκ·€ μž…λ ₯ 6개λ₯Ό μ‚¬μš©ν–ˆλ‹€. μ„Έ ν˜•μ‹μ˜ μž…λ ₯κ³Ό 생성 μƒν•œμ„ κ°™κ²Œ ν–ˆλ‹€. μ‚¬μ‹€Β·λ…ν•΄λŠ” 첫 닡을 μ •κ·œν™”ν•΄ ν—ˆμš© λ‹΅κ³Ό λΉ„κ΅ν–ˆλ‹€. μ‚°μˆ˜λŠ” μš”κ΅¬ν•œ μ •μˆ˜κ°€ 첫 쀄에 λ‹΅μœΌλ‘œ λ‚˜μ™”λŠ”μ§€ μ±„μ ν–ˆλ‹€. λ¬Έν•­ μˆ˜κ°€ μž‘μœΌλ―€λ‘œ ν•œ λ¬Έν•­ 차이둜 ν˜•μ‹μ˜ μš°μ—΄μ„ μ •ν•˜κΈ° μ–΄λ ΅λ‹€.
99
+
100
+ | ν˜•μ‹ | ν•œκ΅­μ–΄Β·μ˜μ–΄ 사싀 (각 12) | ν•œκ΅­μ–΄Β·μ˜μ–΄ 독해 (각 8) | 기본·두 단계 μ‚°μˆ˜ (각 16) | ν•¨μˆ˜ 전체 톡과, μ„€λͺ…λ§Œ / μ˜ˆμ‹œ+λ°˜ν™˜λ¬Έ (각 12) | BoolQ μ •λ‹΅ / 1,404 | BoolQ macro-F1 |
101
+ |---|---:|---:|---:|---:|---:|---:|
102
+ | BF16 | 10 Β· 10 | 7 Β· 7 | 15 Β· 0 | 0 / 9 | 730 (52.0%) | 0.413 |
103
+ | Q8_0 | 10 Β· 10 | 8 Β· 7 | 15 Β· 0 | 0 / 9 | 729 (51.9%) | 0.410 |
104
+ | Q4_K_M | 10 Β· 10 | 7 Β· 7 | 14 Β· 1 | 0 / 9 | 733 (52.2%) | 0.410 |
105
+
106
+ 두 μ˜ˆμ‹œ ν•¨μˆ˜μ™€ λŒ€μƒ ν•¨μˆ˜μ˜ `return`κΉŒμ§€ μž…λ ₯ν•œ 쑰건은 같은 12개 ν•¨μˆ˜μ˜ ν‘œν˜„μ‹ μ™„μ„± λ¬Έμ œλ‹€. λͺ¨λΈμ΄ λ‚˜λ¨Έμ§€ λ³Έλ¬Έ 전체λ₯Ό μž‘μ„±ν•˜λŠ” 쑰건보닀 쉽닀. 좜λ ₯의 λ“€μ—¬μ“°κΈ°λ₯Ό μˆ˜λ¦¬ν•˜μ§€ μ•Šκ³  첫 ν•¨μˆ˜λ§Œ μΆ”μΆœν•΄ νŒŒμ‹±ν•˜κ³ , ν•¨μˆ˜λ‹Ή 4개 독립 μ‚¬λ‘€μ˜ κ°’κ³Ό λ°˜ν™˜ νƒ€μž…μ„ κ²€μ‚¬ν–ˆλ‹€. μ„Έ ν˜•μ‹ λͺ¨λ‘ κ°€μž₯ 큰 포유λ₯˜λ₯Ό 코끼리라고 λ‹΅ν–ˆλ‹€. μ‚°μˆ˜ 총 정닡은 λͺ¨λ‘ 15/32μ˜€μ§€λ§Œ 정닡을 λ‚Έ 문항이 일뢀 λ‹¬λžλ‹€.
107
+
108
+ ν•œκ΅­μ–΄ KoBEST BoolQλŠ” ν…ŒμŠ€νŠΈ 1,404개 μ „μ²΄μ—μ„œ μ§€λ¬ΈΒ·μ§ˆλ¬Έ λ’€ 단일 토큰인 β€˜ κ±°μ§“β€™Β·β€˜ μ°Έβ€™μ˜ 쑰건뢀 λ‘œκ·Έν™•λ₯ μ„ λΉ„κ΅ν•œ zero-shot μ‹œν—˜μ΄λ‹€. BF16μ—λŠ” 같은 배포 κ°€μ€‘μΉ˜μ˜ Transformers BF16 forward, GGUFμ—λŠ” ν•΄λ‹Ή 파일의 llama.cpp 채점기λ₯Ό μ‚¬μš©ν–ˆλ‹€. λ‹€μˆ˜ ν΄λž˜μŠ€λŠ” μ°Έ 705/1,404(50.2%)λ‹€. λ‹€μŒ ν‘œμ˜ μ°Έ 예츑 λΉˆλ„μ™€ macro-F1을 ν•¨κ»˜ ν•΄μ„ν•œλ‹€. 데이터 λΌμ΄μ„ μŠ€λŠ” CC BY-SA 4.0이며 μ‹œν—˜μ˜ ν•™μŠ΅ 데이터 쀑볡 μ—¬λΆ€λŠ” 미확인이닀. [KoBEST κ³ μ • revision](https://huggingface.co/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f)
109
+
110
+ | ν˜•μ‹ | BoolQ μ°Έ 예츑 수 | BF16 λŒ€λΉ„ λ§žμŒβ†’ν‹€λ¦Ό / ν‹€λ¦Όβ†’λ§žμŒ | ν•œκ΅­μ–΄ / μ˜μ–΄ 이야기 반볡 4-gram | ν•œκ΅­μ–΄ / μ˜μ–΄μ—μ„œ μž₯면을 μ „κ°œν•œ 좜λ ₯ (각 4개) |
111
+ |---|---:|---:|---:|---:|
112
+ | BF16 | 1299 | 0 / 0 | 53.7% / 58.6% | 1 / 1 |
113
+ | Q8_0 | 1302 | 5 / 4 | 42.6% / 61.9% | 2 / 0 |
114
+ | Q4_K_M | 1312 | 19 / 22 | 66.4% / 68.4% | 0 / 0 |
115
+
116
+ 반볡λ₯ μ€ 64토큰 생성물을 λ‹€μ‹œ 토큰화해 처음 λ“±μž₯ν•œ μ΄ν›„μ˜ 쀑볡 4-gram 수λ₯Ό 전체 4-gram 수둜 λ‚˜λˆˆ 값이닀. 이어쓰기 8κ°œλŠ” ν˜•μ‹λͺ…을 가리고 같은 μž…λ ₯·같은 좜λ ₯을 합쳐 단일 AI νŒλ…μžκ°€ 0–2점으둜 λΆ„λ₯˜ν–ˆλ‹€. ν‘œμ—λŠ” μ„œλ‘œ λ‹€λ₯Έ μ„ΈλΆ€ λ‚΄μš©μœΌλ‘œ μž₯면이 μ „κ°œλœ 2점 좜λ ₯ 수λ₯Ό ν‘œμ‹œν–ˆλ‹€. 예λ₯Ό λ“€μ–΄ λΉ„κ°€ 그친 μš΄λ™μž₯μ΄λΌλŠ” μ˜μ–΄ μž…λ ₯μ—μ„œλŠ” μž”λ”” λ¬˜μ‚¬κ°€ 같은 λ¬Έμž₯으둜 λ°˜λ³΅λλ‹€. μ‚¬λžŒ ν‰κ°€μž κ°„ μΌμΉ˜λ„λŠ” μΈ‘μ •ν•˜μ§€ μ•Šμ•˜λ‹€.
117
+
118
+ ### 언어별 holdout PPL
119
+
120
+ κ²€μ¦λœ 1,466λ¬Έμ„œμ˜ 동일 토큰 IDΒ·λ§ˆμŠ€ν¬Β·λ¬Έμ„œλ³„ NLL 합을 μž¬μ‚¬μš©ν–ˆλ‹€. ν•œκ΅­μ–΄ 663,905개, μ˜μ–΄ 705,634개 ν‘œμ μ΄λ‹€. κ΄„ν˜Έ μ•ˆμ€ BF16 λŒ€λΉ„ μƒλŒ€ μƒμŠΉλ₯ μ˜ 95% ꡬ간이닀. λ¬Έμ„œλ₯Ό μ–Έμ–΄λ³„λ‘œ 짝지어 2,000회 볡원 μΆ”μΆœν•˜κ³  맀번 ν‘œμ  토큰 가쀑 평균 NLL둜 PPL을 κ΅¬ν–ˆλ‹€.
121
+
122
+ | ν˜•μ‹ | ν•œκ΅­μ–΄ PPL | ν•œκ΅­μ–΄ μƒμŠΉλ₯  (95% ꡬ간) | μ˜μ–΄ PPL | μ˜μ–΄ μƒμŠΉλ₯  (95% ꡬ간) |
123
+ |---|---:|---:|---:|---:|
124
+ | BF16 | 9.08981 | κΈ°μ€€ | 10.21574 | κΈ°μ€€ |
125
+ | Q8_0 | 9.09086 | +0.012% (+0.002 ~ +0.022%) | 10.21729 | +0.015% (+0.005 ~ +0.025%) |
126
+ | Q4_K_M | 9.19630 | +1.172% (+1.110 ~ +1.235%) | 10.31774 | +0.999% (+0.952 ~ +1.042%) |
127
+
128
+ ### λ¬Έλ§₯ 길이와 λ©”λͺ¨λ¦¬
129
+
130
+ μ„œλ²„λ₯Ό μ˜ˆμ—΄ν•œ λ’€ μž…λ ₯ 길이·좜λ ₯ μƒν•œλ³„λ‘œ 5νšŒμ”© μˆœμ„œλ₯Ό μ„žμ—ˆλ‹€. μ„±λŠ₯ μž…λ ₯은 ν•œ 단어당 ν•œ 토큰인 κ³ μ • μ–΄νœ˜μ—μ„œ seed둜 λ½‘μ•˜μœΌλ©° ν•œκ΅­μ–΄ ν’ˆμ§ˆ μ‹œν—˜ μž…λ ₯κ³Ό 별도닀. 각 ν˜•μ‹μ— 같은 ν”„λ‘¬ν”„νŠΈλ₯Ό μ „μ†‘ν–ˆκ³  성곡 μ‘λ‹΅μ˜ μž…λ ₯·좜λ ₯ 토큰 수λ₯Ό μ „μˆ˜ ν™•μΈν–ˆλ‹€. μ•„λž˜λŠ” 좜λ ₯ 128토큰, 단일 μš”μ²­μ˜ 쀑앙값이닀. 첫 ν…μŠ€νŠΈκ°€ 도착할 λ•ŒκΉŒμ§€κ°€ TTFT이고 생성 μ†λ„λŠ” 이후 남은 토큰을 μ™„λ£Œν•  λ•ŒκΉŒμ§€μ˜ 속도닀. 졜고 VRAM은 μš”μ²­ 경��와 0.5초 κ°„κ²©μ—μ„œ κ΄€μΈ‘ν•œ μž₯치 전체 μ‚¬μš©λŸ‰μ΄λ‹€.
131
+
132
+ | μž…λ ₯ 토큰 | ν˜•μ‹ | 성곡 / μ‹œλ„ | TTFT (ms) | 전체 μ§€μ—° (초) | 생성 토큰/초 | 졜고 VRAM (MiB) |
133
+ |---:|---|---:|---:|---:|---:|---:|
134
+ | 128 | BF16 | 5/5 | 19.7 | 1.280 | 100.8 | 8805.9 |
135
+ | 128 | Q8_0 | 5/5 | 65.9 | 0.445 | 335.2 | 5145.9 |
136
+ | 128 | Q4_K_M | 5/5 | 65.5 | 0.418 | 361.2 | 3851.9 |
137
+ | 1,024 | BF16 | 5/5 | 26.2 | 1.290 | 100.5 | 8805.9 |
138
+ | 1,024 | Q8_0 | 5/5 | 112.7 | 0.490 | 335.4 | 5145.9 |
139
+ | 1,024 | Q4_K_M | 5/5 | 131.5 | 0.483 | 362.0 | 3851.9 |
140
+ | 3,072 | BF16 | 5/5 | 57.6 | 1.321 | 100.5 | 8805.9 |
141
+ | 3,072 | Q8_0 | 5/5 | 265.8 | 0.663 | 320.2 | 5145.9 |
142
+ | 3,072 | Q4_K_M | 5/5 | 292.1 | 0.666 | 343.2 | 3851.9 |
143
+ | 3,968 | BF16 | 5/5 | 72.3 | 1.334 | 100.8 | 8805.9 |
144
+ | 3,968 | Q8_0 | 5/5 | 362.4 | 0.753 | 324.8 | 5145.9 |
145
+ | 3,968 | Q4_K_M | 4/5 | 404.1 | 0.768 | 349.0 | 3851.9 |
146
+
147
+ Q4_K_M은 3,968+128 μ‘°κ±΄μ—μ„œ ν•œ μš”μ²­μ΄ 반볡 토큰 μ œν•œμœΌλ‘œ μ’…λ£ŒλμœΌλ©° 같은 μž…λ ₯의 별도 μž¬ν˜„μ—μ„œλ„ 같은 였λ₯˜κ°€ λ°œμƒν–ˆλ‹€. 이 ν–‰μ˜ μ§€μ—°Β·μ†λ„λŠ” μ„±κ³΅ν•œ 4회둜 κ³„μ‚°ν–ˆκ³  VRAMμ—λŠ” μ‹€νŒ¨ μš”μ²­λ„ ν¬ν•¨ν–ˆλ‹€. 4,096은 μž…λ ₯κ³Ό 좜λ ₯을 ν•©ν•œ ν•œλ„λ‹€. μ„œλ²„κ°€ μ˜ˆμ—΄λœ μƒνƒœμ—μ„œ KV μΊμ‹œμ™€ μ‹€ν–‰ 버퍼λ₯Ό μž¬μ‚¬μš©ν•˜λ―€λ‘œ μž…λ ₯ 길이가 λ°”λ€Œμ–΄λ„ κ΄€μΈ‘ 졜고 VRAM이 λΉ„μŠ·ν•  수 μžˆλ‹€.
148
+
149
+ ### λ™μ‹œ μš”μ²­κ³Ό ν˜•μ‹ 선택
150
+
151
+ μž…λ ₯ 1,024ν† ν°Β·μΆœλ ₯ 32ν† ν°μ—μ„œ λ™μ‹œμ— 1Β·2Β·4개 μš”μ²­μ„ λ³΄λƒˆλ‹€. ν‘œμ˜ μ²˜λ¦¬λŸ‰μ€ μš”μ²­ 묢음 μ „μ²΄μ˜ 좜λ ₯ 토큰 수λ₯Ό 첫 μš”μ²­ μ‹œμž‘λΆ€ν„° λ§ˆμ§€λ§‰ μ™„λ£ŒκΉŒμ§€μ˜ μ‹œκ°„μœΌλ‘œ λ‚˜λˆˆ κ°’μ˜ 쀑앙값이닀. OllamaλŠ” μ„œλ²„ 병렬 슬둯 1개둜 큐 λŒ€κΈ°λ₯Ό ν¬ν•¨ν•œλ‹€. vLLM은 μ΅œλŒ€ 2개 μ‹œν€€μŠ€λ₯Ό ν•¨κ»˜ μ²˜λ¦¬ν•œλ‹€.
152
+
153
+ | ν˜•μ‹ | λ™μ‹œ 1 / 2 / 4 전체 μ²˜λ¦¬λŸ‰ (토큰/초) | λ™μ‹œ 4 μš”μ²­ μ§€μ—° 쀑앙값 / κ΄€μΈ‘ μ΅œλŒ“κ°’ (초) |
154
+ |---|---:|---:|
155
+ | BF16 | 95.0 / 179.3 / 185.1 | 0.517 / 0.690 |
156
+ | Q8_0 | 102.2 / 143.9 / 140.8 | 0.586 / 0.990 |
157
+ | Q4_K_M | 102.2 / 144.9 / 142.0 | 0.584 / 0.975 |
158
+
159
+ ν˜•μ‹λ³„ ꢌμž₯ μ‹œμž‘ 섀정은 λ‹€μŒκ³Ό κ°™λ‹€. 속도 λΉ„κ΅μ—λŠ” κ°€μ€‘μΉ˜ 정밀도뿐 μ•„λ‹ˆλΌ λŸ°νƒ€μž„Β·μ»€λ„Β·μΊμ‹œ μ„€μ • 차이도 ν¬ν•¨λœλ‹€.
160
+
161
+ - **BF16 Β· 정확도 비ꡐ κΈ°μ€€, 짧은 첫 응닡 λ˜λŠ” λ™μ‹œ μš”μ²­.** H200 1μž₯에 `--dtype bfloat16 --load-format safetensors --tensor-parallel-size 1 --max-model-len 4096 --max-num-seqs 2 --max-num-batched-tokens 4096 --gpu-memory-utilization 0.25 --num-gpu-blocks-override 512 --enforce-eager --no-enable-prefix-caching --generation-config vllm --seed 47000`을 μ‚¬μš©ν–ˆλ‹€. `VLLM_ATTENTION_BACKEND=FLASH_ATTN`κ³Ό `VLLM_USE_FLASHINFER_SAMPLER=0`을 μ μš©ν–ˆλ‹€. `/v1/completions`둜 `temperature=0`κ³Ό λͺ©μ μ— λ§žλŠ” `max_tokens`λ₯Ό μ „λ‹¬ν•œλ‹€.
162
+ - **Q8_0 Β· Ollama의 ν’ˆμ§ˆ μš°μ„  κΈ°λ³Έκ°’.** BF16에 κ°€κΉŒμš΄ PPL이 ν•„μš”ν•œ 경우 `hf.co/pathcosmos/JANSORI:Q8_0`을 μ„ νƒν•œλ‹€. `/api/generate`μ—μ„œ `raw=true`, `stream=true`, `num_ctx=4096`, `num_predict=32`(짧은 λ‹΅) λ˜λŠ” `128`(이어쓰기), `temperature=0`, `repeat_penalty=1`, `seed=47000`을 μ‚¬μš©ν•œλ‹€. 이 H200 μ‹œν—˜μ€ `num_gpu=99`, `num_thread=4`, `OLLAMA_NUM_PARALLEL=1`μ΄μ—ˆλ‹€.
163
+ - **Q4_K_M Β· λ©”λͺ¨λ¦¬ μš°μ„ .** 같은 Ollama μ˜΅μ…˜μ— νƒœκ·Έλ₯Ό `Q4_K_M`으둜 λ°”κΎΌλ‹€. κΈ΄ μž…λ ₯μ—μ„œλŠ” μ΅œλŒ€ 길이보닀 μ—¬μœ λ₯Ό 두고 생성 μƒν•œμ„ λͺ…μ‹œν•œλ‹€. 이번 3,072+128 μ‹œν—˜μ€ 5/5 μ™„λ£Œλλ‹€. Q4κ°€ λͺ¨λ“  μž…λ ₯ κΈΈμ΄μ—μ„œ κ°€μž₯ λΉ λ₯΄λ‹€λŠ” 보μž₯은 μ—†λ‹€.
164
+ - **μ‚°μˆ˜Β·μ½”λ“œ.** 짧은 정닡을 μš”κ΅¬ν•˜λŠ” λ‹¨μˆœ μ‚°μˆ˜μ™€ 두 단계 μ‚°μˆ˜ κ²°κ³Όλ₯Ό κ΅¬λΆ„ν•œλ‹€. κ°„λ‹¨ν•œ Python ν•¨μˆ˜ μ™„μ„±μ—λŠ” ν•¨μˆ˜ μ˜ˆμ‹œ 두 κ°œμ™€ λͺ©ν‘œ ν•¨μˆ˜μ˜ `return` 접두어λ₯Ό μ μš©ν•œ λ’€ 별도 ν…ŒμŠ€νŠΈλ‘œ κ²€μ¦ν•œλ‹€. 이 κ²°κ³ΌλŠ” μž„μ˜ ν”„λ‘œκ·Έλž¨ 생성 λŠ₯λ ₯을 ν‰κ°€ν•˜μ§€ μ•ŠλŠ”λ‹€.
165
+ ### H200 두 μž₯μ—μ„œ μš”μ²­μ„ λ³‘ν–‰ν•œ 경우
166
+
167
+ GPU 0에 BF16, GPU 1에 Ollama ν•œ ν˜•μ‹μ„ μ μž¬ν•œ λ’€ 1,024토큰 μž…λ ₯κ³Ό 32토큰 좜λ ₯을 두 μ„œλ²„μ— 각각 μš”μ²­ν–ˆλ‹€. ν˜•μ‹ μ‘°ν•©λ³„λ‘œ μ„œλ‘œ λ‹€λ₯Έ μž…λ ₯ 5개λ₯Ό μ‚¬μš©ν–ˆλ‹€. 같은 μž…λ ₯을 μˆœμ°¨μ™€ 병행에 λͺ¨λ‘ μ‚¬μš©ν•˜κ³  μ–΄λŠ μͺ½μ„ λ¨Όμ € μΈ‘μ •ν• μ§€λŠ” κ΅λŒ€ν–ˆλ‹€. 각 μΈ‘μ • μ „ 별도 1,024토큰 μš”μ²­μœΌλ‘œ 직전 μž…λ ₯ μΊμ‹œκ°€ κ·ΈλŒ€λ‘œ μž¬μ‚¬μš©λ˜λŠ” 경우λ₯Ό μ€„μ˜€λ‹€. μ„œλ²„ μ‹œμž‘Β·λͺ¨λΈ λ‘œλ”©μ€ 이 ν‘œμ—μ„œ μ œμ™Έν–ˆλ‹€.
168
+
169
+ | λ™μ‹œ 상주 ν˜•μ‹ | 순차 / 병행 묢음 μ§€μ—° 쀑앙값 (초) | 병행 μš”μ²­ 쀑첩 쀑앙값 (초) | 두 좜λ ₯ λͺ¨λ‘ λ™μΌν•œ μž…λ ₯ | 동일 좜λ ₯ 쌍만의 순차 / 병행 쀑앙값 (초) | 졜고 λ™μ‹œ ν•©μ‚° VRAM, 순차 / 병행 (MiB) |
170
+ |---|---:|---:|---:|---:|---:|
171
+ | BF16 + Q8_0 | 0.568 / 0.323 | 0.240 | 4/5 | 0.551 / 0.324 | 13889.8 / 13889.8 |
172
+ | BF16 + Q4_K_M | 0.556 / 0.324 | 0.235 | 3/5 | 0.559 / 0.324 | 12595.8 / 12595.8 |
173
+
174
+ μ–‘μͺ½ λͺ¨λΈμ΄ μƒμ£Όν•œ μƒνƒœμ΄λ―€λ‘œ 순차 μš”μ²­λ„ 두 λͺ¨λΈμ˜ VRAM을 μœ μ§€ν•œλ‹€. 동일 좜λ ₯ 쌍 μˆ˜κ°€ 적을 λ•ŒλŠ” 속도 차이λ₯Ό 동일 ν’ˆμ§ˆμ—μ„œ 보μž₯λ˜λŠ” κ°œμ„ μœΌλ‘œ ν•΄μ„ν•˜κΈ° μ–΄λ ΅λ‹€. κΈ°μ‘΄ 곡개 κ²€μ¦μ˜ μ„œλ²„ μ‹œμž‘λΆ€ν„° μ’…λ£ŒκΉŒμ§€ 잰 μˆ˜μΉ˜λŠ” λ‹€λ₯Έ μ§€ν‘œλ‹€.
175
+
176
  ## ν•™μŠ΅ 데이터
177
 
178
  ν•œκ΅­μ–΄ 쀑심 3B Dense Base λͺ¨λΈμ€ 47,000μŠ€ν…μ—μ„œ λˆ„μ  98,566,144,000토큰을 ν•™μŠ΅ν–ˆλ‹€. μ•„λž˜ μˆ˜μΉ˜λŠ” ν•™μŠ΅ μ‹œ 반볡 λ…ΈμΆœμ„ ν¬ν•¨ν•œ μΆœμ²˜λ³„ μž…λ ₯ 토큰 μˆ˜λ‹€.
SHA256SUMS CHANGED
@@ -1,7 +1,7 @@
1
  e656e13897e3ef9c16a5bd506b16cc1cd557212bd0c55e10994be2e70e01bf47 .gitattributes
2
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
3
  057f847c2bb770535d520ca66c61eff6de69f8095385cf1d5f69f48b471f96d6 NOTICE
4
- ea1c10074d26dd9b3eac8978d03ac289056cf4673ef03fe66faebcb1cf603148 README.md
5
  b94eb89d477bb22489a5730b77ea4cdf92af9c63dbddc3dc5cd7967e87d4749e config.json
6
  21f8c90ade582967a8acb1991f73bea5f62757ae2d430500da41ba458f0f52a0 dense-3b-47k-Q4_K_M.gguf
7
  eb7053112886dab60d6012d4635dc2c7cdd07c902317e202a91ba36da33fed41 dense-3b-47k-Q8_0.gguf
 
1
  e656e13897e3ef9c16a5bd506b16cc1cd557212bd0c55e10994be2e70e01bf47 .gitattributes
2
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
3
  057f847c2bb770535d520ca66c61eff6de69f8095385cf1d5f69f48b471f96d6 NOTICE
4
+ 57b502f333dc02f6fb6064a6cae1626876501ebc7f6a6f9fd950da9bb3d15d91 README.md
5
  b94eb89d477bb22489a5730b77ea4cdf92af9c63dbddc3dc5cd7967e87d4749e config.json
6
  21f8c90ade582967a8acb1991f73bea5f62757ae2d430500da41ba458f0f52a0 dense-3b-47k-Q4_K_M.gguf
7
  eb7053112886dab60d6012d4635dc2c7cdd07c902317e202a91ba36da33fed41 dense-3b-47k-Q8_0.gguf