kmamaroziqov commited on
Commit
b2e14ee
·
verified ·
1 Parent(s): c183ccc

docs: add classification usage section (news 10-way + binary sentiment)

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md CHANGED
@@ -176,6 +176,108 @@ enough for inference.
176
  `config.json` sets `use_cache: false`, but `generation_config.json` sets `use_cache: true`, so
177
  `generate()` uses the KV cache. Pass `use_cache=True` explicitly if you write your own decode loop.
178
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
179
  ### Serving
180
 
181
  **vLLM and SGLang cannot load this model.** They reimplement each architecture
 
176
  `config.json` sets `use_cache: false`, but `generation_config.json` sets `use_cache: true`, so
177
  `generate()` uses the KV cache. Pass `use_cache=True` explicitly if you write your own decode loop.
178
 
179
+ ### Classification
180
+
181
+ The model is usable as a constrained label picker: put the label set in the prompt, ask
182
+ for the label only, decode **greedily**, and cap `max_new_tokens`. Terse tasks showed a
183
+ 0% repetition rate under every decoding configuration tested, so no repetition penalty
184
+ is needed here — and greedy keeps the output reproducible.
185
+
186
+ These are the exact prompts behind the news (0.6531) and sentiment (0.9259) scores in
187
+ [Evaluation](#uzbek-benchmarks). Reuse them verbatim to reproduce those numbers.
188
+
189
+ ```python
190
+ import re
191
+ import torch
192
+ from transformers import AutoModelForCausalLM, AutoTokenizer
193
+
194
+ model_id = "NeuronUz/MustaqiLLM"
195
+
196
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
197
+ model = AutoModelForCausalLM.from_pretrained(
198
+ model_id,
199
+ trust_remote_code=True,
200
+ dtype=torch.bfloat16,
201
+ device_map="cuda",
202
+ ).eval()
203
+
204
+
205
+ def classify(prompt: str, text: str, max_chars: int = 4000) -> str:
206
+ if len(text) > max_chars:
207
+ text = text[:max_chars].rsplit(" ", 1)[0]
208
+ inputs = tokenizer.apply_chat_template(
209
+ [{"role": "user", "content": prompt.format(text=text)}],
210
+ add_generation_prompt=True,
211
+ return_tensors="pt",
212
+ return_dict=True,
213
+ ).to(model.device)
214
+
215
+ with torch.no_grad():
216
+ out = model.generate(
217
+ **inputs,
218
+ max_new_tokens=12, # a label is a few tokens; do not give it room to ramble
219
+ do_sample=False, # greedy -- labels must be deterministic
220
+ pad_token_id=3, # <pad>
221
+ )
222
+ return tokenizer.decode(
223
+ out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True
224
+ ).strip()
225
+ ```
226
+
227
+ **News topic, 10-way.** Numbered labels: one digit is easier to emit and to parse than a
228
+ multi-word category name.
229
+
230
+ ```python
231
+ NEWS_LABELS = [
232
+ "Siyosat", "Iqtisodiyot", "Texnologiya", "Sport", "Madaniyat",
233
+ "Salomatlik", "Oila va Jamiyat", "Ta'lim", "Ekologiya", "Xorijiy Yangiliklar",
234
+ ]
235
+
236
+ NEWS_PROMPT = (
237
+ "Classify the given Uzbek news article into one of the following categories. "
238
+ "Respond with only the category number.\n\n"
239
+ + "".join(f"{i} - {name}\n" for i, name in enumerate(NEWS_LABELS))
240
+ + "\nArticle: {text}\n\nAnswer:"
241
+ )
242
+
243
+ raw = classify(NEWS_PROMPT, "O'zbekiston Markaziy banki asosiy stavkani o'zgarishsiz qoldirdi.")
244
+ match = re.search(r"\d+", raw)
245
+ label = NEWS_LABELS[int(match.group())] if match and int(match.group()) < 10 else None
246
+ print(raw, "->", label)
247
+ ```
248
+
249
+ ```
250
+ 1 -> Iqtisodiyot
251
+ ```
252
+
253
+ **Sentiment, binary.**
254
+
255
+ ```python
256
+ SENTIMENT_PROMPT = (
257
+ "Given the following Uzbek text, determine the sentiment as either "
258
+ "'Positive' or 'Negative'. Respond with only one label.\n\n"
259
+ "Text: {text}\n\nLabel:"
260
+ )
261
+
262
+ raw = classify(SENTIMENT_PROMPT, "Mahsulot juda sifatli, yetkazib berish tez bo'ldi.")
263
+ print(raw) # Positive
264
+ ```
265
+
266
+ **Your own label set.** The same shape works for any closed label set — put one label
267
+ per line, demand the label (or its number) and nothing else, and parse the output with a
268
+ prefix match or a regex rather than an exact-string comparison, so a stray token never
269
+ becomes an invalid prediction. Two practical notes:
270
+
271
+ - **A task-specific system prompt is fine here** and often helps — it is the generic
272
+ "you are a helpful assistant" turn that degrades output (see
273
+ [Generation settings](#generation-settings)). Put the required output format in it.
274
+ - **English prompt text with Uzbek labels** is what was measured. Uzbek prompt wording
275
+ also works; if you change the wording, re-measure — label boundaries (especially
276
+ `Siyosat` vs `Xorijiy Yangiliklar`, and `Oila va Jamiyat`, the weakest class at 0.4273)
277
+ are sensitive to how the categories are described.
278
+ - **Do not batch-compare greedy runs at different batch sizes.** Left-padding shifts the
279
+ numerics; identical prompts matched in only 24 of 32 cases between batch 1 and batch 12.
280
+
281
  ### Serving
282
 
283
  **vLLM and SGLang cannot load this model.** They reimplement each architecture