gyung commited on
Commit
c8febc2
ยท
verified ยท
1 Parent(s): d5e64d3

Update GGUF README with H/L cycle override instructions

Browse files
Files changed (1) hide show
  1. README.md +44 -0
README.md CHANGED
@@ -86,6 +86,50 @@ Run on CPU:
86
  -p $'ํ•ด์™ธ์ฃผ์‹ ํˆฌ์ž์—์„œ ์›/๋‹ฌ๋Ÿฌ ํ™˜์œจ ๋ณ€๋™์ด ์›ํ™” ์ˆ˜์ต๋ฅ ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ๊ณผ ๋Œ€์‘ ๋ฐฉ์•ˆ์„ ๊ฐ„๋‹จํžˆ ์„ค๋ช…ํ•ด ์ฃผ์„ธ์š”.'
87
  ```
88
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
  ## CPU Generation Tests
90
 
91
  Tested locally on CPU with the patched `llama.cpp` build and the prompt shown above.
 
86
  -p $'ํ•ด์™ธ์ฃผ์‹ ํˆฌ์ž์—์„œ ์›/๋‹ฌ๋Ÿฌ ํ™˜์œจ ๋ณ€๋™์ด ์›ํ™” ์ˆ˜์ต๋ฅ ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ๊ณผ ๋Œ€์‘ ๋ฐฉ์•ˆ์„ ๊ฐ„๋‹จํžˆ ์„ค๋ช…ํ•ด ์ฃผ์„ธ์š”.'
87
  ```
88
 
89
+ ## H/L Cycle Override (์ˆ˜์ • ์‹คํ–‰: H/L ์‚ฌ์ดํด ์ง์ ‘ ์ง€์ •)
90
+
91
+ KoHRM-Text-GGUF stores recurrence settings as GGUF metadata:
92
+
93
+ ```text
94
+ hrm_text.h_cycles = 2
95
+ hrm_text.l_cycles = 3
96
+ ```
97
+
98
+ ํ˜„์žฌ ํŒจ์น˜๋œ llama.cpp์—์„œ๋Š” ์‹คํ–‰ ์‹œ ๋ชจ๋ธ ๋กœ๋”ฉ ๋‹จ๊ณ„์—์„œ ์•„๋ž˜ ํ‚ค๋ฅผ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์˜ค๋ฒ„๋ผ์ด๋“œํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
99
+
100
+ ```bash
101
+ ./build-hrm/bin/llama-cli \
102
+ -m ./KoHRM-Text-1.4B-Q8_0.gguf \
103
+ -ngl 0 \
104
+ -t 4 \
105
+ -c 1024 \
106
+ -n 260 \
107
+ --seed 41 \
108
+ --temp 0.45 \
109
+ --top-p 0.9 \
110
+ --repeat-penalty 1.08 \
111
+ --single-turn \
112
+ --simple-io \
113
+ --no-warmup \
114
+ --display-prompt \
115
+ --override-kv hrm_text.h_cycles=int:1 \
116
+ --override-kv hrm_text.l_cycles=int:2 \
117
+ -p $'ํ•ด์™ธ์ฃผ์‹ ํˆฌ์ž์—์„œ ์›/๋‹ฌ๋Ÿฌ ํ™˜์œจ ๋ณ€๋™์ด ์›ํ™” ์ˆ˜์ต๋ฅ ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ๊ณผ ๋Œ€์‘ ๋ฐฉ์•ˆ์„ ๊ฐ„๋‹จํžˆ ์„ค๋ช…ํ•ด ์ฃผ์„ธ์š”.'
118
+ ```
119
+
120
+ Use case:
121
+
122
+ - `h_cycles/l_cycles`๋ฅผ ๋‚ฎ์ถ”๋ฉด ๋™์ผ ์กฐ๊ฑด์—์„œ ์‘๋‹ต์†๋„๋Š” ๋นจ๋ผ์ง€๋Š” ๊ฒฝํ–ฅ์ด ์žˆ์ง€๋งŒ ํ’ˆ์งˆ ์ €ํ•˜๊ฐ€ ์ž์ฃผ ์ฆ๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
123
+ - `2/3`์€ ํ˜„์žฌ ๊ธฐ๋ณธ๊ฐ’(๋ฌธ์„œ ์ƒ ์•ˆ์ • ๋™์ž‘)์ž…๋‹ˆ๋‹ค.
124
+ - `1/2`๋Š” ์†๋„ ์šฐ์„  ํ…Œ์ŠคํŠธ๋กœ ์ถ”์ฒœํ•˜๋ฉฐ, ์‹ค์ œ ์ •๋ฐ€ ์ถ”๋ก ์—์„œ๋Š” `2/3`์ด ๋” ์•ˆ์ •์ ์ž…๋‹ˆ๋‹ค.
125
+
126
+ `--override-kv` uses key format `KEY=TYPE:VALUE`, same as upstream llama.cpp:
127
+
128
+ - `hrm_text.h_cycles=int:1`
129
+ - `hrm_text.l_cycles=int:2`
130
+
131
+ If you need a persistent configuration (e.g., fixed 1/1 for a workload), export a new GGUF after changing `H_cycles` / `L_cycles` in the source config before `convert_hf_to_gguf.py` conversion. That preserves one set of cycles inside the artifact and avoids runtime override overhead.
132
+
133
  ## CPU Generation Tests
134
 
135
  Tested locally on CPU with the patched `llama.cpp` build and the prompt shown above.