Text Generation
Transformers
Safetensors
Uzbek
English
Russian
neuron_lm
uzbek
o'zbek
chat
instruction-tuned
conversational
custom_code
Instructions to use NeuronUz/MustaqiLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use NeuronUz/MustaqiLLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="NeuronUz/MustaqiLLM", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("NeuronUz/MustaqiLLM", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use NeuronUz/MustaqiLLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "NeuronUz/MustaqiLLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NeuronUz/MustaqiLLM", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/NeuronUz/MustaqiLLM
- SGLang
How to use NeuronUz/MustaqiLLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "NeuronUz/MustaqiLLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NeuronUz/MustaqiLLM", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "NeuronUz/MustaqiLLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NeuronUz/MustaqiLLM", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use NeuronUz/MustaqiLLM with Docker Model Runner:
docker model run hf.co/NeuronUz/MustaqiLLM
docs: add classification usage section (news 10-way + binary sentiment)
Browse files
README.md
CHANGED
|
@@ -176,6 +176,108 @@ enough for inference.
|
|
| 176 |
`config.json` sets `use_cache: false`, but `generation_config.json` sets `use_cache: true`, so
|
| 177 |
`generate()` uses the KV cache. Pass `use_cache=True` explicitly if you write your own decode loop.
|
| 178 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
### Serving
|
| 180 |
|
| 181 |
**vLLM and SGLang cannot load this model.** They reimplement each architecture
|
|
|
|
| 176 |
`config.json` sets `use_cache: false`, but `generation_config.json` sets `use_cache: true`, so
|
| 177 |
`generate()` uses the KV cache. Pass `use_cache=True` explicitly if you write your own decode loop.
|
| 178 |
|
| 179 |
+
### Classification
|
| 180 |
+
|
| 181 |
+
The model is usable as a constrained label picker: put the label set in the prompt, ask
|
| 182 |
+
for the label only, decode **greedily**, and cap `max_new_tokens`. Terse tasks showed a
|
| 183 |
+
0% repetition rate under every decoding configuration tested, so no repetition penalty
|
| 184 |
+
is needed here — and greedy keeps the output reproducible.
|
| 185 |
+
|
| 186 |
+
These are the exact prompts behind the news (0.6531) and sentiment (0.9259) scores in
|
| 187 |
+
[Evaluation](#uzbek-benchmarks). Reuse them verbatim to reproduce those numbers.
|
| 188 |
+
|
| 189 |
+
```python
|
| 190 |
+
import re
|
| 191 |
+
import torch
|
| 192 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 193 |
+
|
| 194 |
+
model_id = "NeuronUz/MustaqiLLM"
|
| 195 |
+
|
| 196 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
|
| 197 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 198 |
+
model_id,
|
| 199 |
+
trust_remote_code=True,
|
| 200 |
+
dtype=torch.bfloat16,
|
| 201 |
+
device_map="cuda",
|
| 202 |
+
).eval()
|
| 203 |
+
|
| 204 |
+
|
| 205 |
+
def classify(prompt: str, text: str, max_chars: int = 4000) -> str:
|
| 206 |
+
if len(text) > max_chars:
|
| 207 |
+
text = text[:max_chars].rsplit(" ", 1)[0]
|
| 208 |
+
inputs = tokenizer.apply_chat_template(
|
| 209 |
+
[{"role": "user", "content": prompt.format(text=text)}],
|
| 210 |
+
add_generation_prompt=True,
|
| 211 |
+
return_tensors="pt",
|
| 212 |
+
return_dict=True,
|
| 213 |
+
).to(model.device)
|
| 214 |
+
|
| 215 |
+
with torch.no_grad():
|
| 216 |
+
out = model.generate(
|
| 217 |
+
**inputs,
|
| 218 |
+
max_new_tokens=12, # a label is a few tokens; do not give it room to ramble
|
| 219 |
+
do_sample=False, # greedy -- labels must be deterministic
|
| 220 |
+
pad_token_id=3, # <pad>
|
| 221 |
+
)
|
| 222 |
+
return tokenizer.decode(
|
| 223 |
+
out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True
|
| 224 |
+
).strip()
|
| 225 |
+
```
|
| 226 |
+
|
| 227 |
+
**News topic, 10-way.** Numbered labels: one digit is easier to emit and to parse than a
|
| 228 |
+
multi-word category name.
|
| 229 |
+
|
| 230 |
+
```python
|
| 231 |
+
NEWS_LABELS = [
|
| 232 |
+
"Siyosat", "Iqtisodiyot", "Texnologiya", "Sport", "Madaniyat",
|
| 233 |
+
"Salomatlik", "Oila va Jamiyat", "Ta'lim", "Ekologiya", "Xorijiy Yangiliklar",
|
| 234 |
+
]
|
| 235 |
+
|
| 236 |
+
NEWS_PROMPT = (
|
| 237 |
+
"Classify the given Uzbek news article into one of the following categories. "
|
| 238 |
+
"Respond with only the category number.\n\n"
|
| 239 |
+
+ "".join(f"{i} - {name}\n" for i, name in enumerate(NEWS_LABELS))
|
| 240 |
+
+ "\nArticle: {text}\n\nAnswer:"
|
| 241 |
+
)
|
| 242 |
+
|
| 243 |
+
raw = classify(NEWS_PROMPT, "O'zbekiston Markaziy banki asosiy stavkani o'zgarishsiz qoldirdi.")
|
| 244 |
+
match = re.search(r"\d+", raw)
|
| 245 |
+
label = NEWS_LABELS[int(match.group())] if match and int(match.group()) < 10 else None
|
| 246 |
+
print(raw, "->", label)
|
| 247 |
+
```
|
| 248 |
+
|
| 249 |
+
```
|
| 250 |
+
1 -> Iqtisodiyot
|
| 251 |
+
```
|
| 252 |
+
|
| 253 |
+
**Sentiment, binary.**
|
| 254 |
+
|
| 255 |
+
```python
|
| 256 |
+
SENTIMENT_PROMPT = (
|
| 257 |
+
"Given the following Uzbek text, determine the sentiment as either "
|
| 258 |
+
"'Positive' or 'Negative'. Respond with only one label.\n\n"
|
| 259 |
+
"Text: {text}\n\nLabel:"
|
| 260 |
+
)
|
| 261 |
+
|
| 262 |
+
raw = classify(SENTIMENT_PROMPT, "Mahsulot juda sifatli, yetkazib berish tez bo'ldi.")
|
| 263 |
+
print(raw) # Positive
|
| 264 |
+
```
|
| 265 |
+
|
| 266 |
+
**Your own label set.** The same shape works for any closed label set — put one label
|
| 267 |
+
per line, demand the label (or its number) and nothing else, and parse the output with a
|
| 268 |
+
prefix match or a regex rather than an exact-string comparison, so a stray token never
|
| 269 |
+
becomes an invalid prediction. Two practical notes:
|
| 270 |
+
|
| 271 |
+
- **A task-specific system prompt is fine here** and often helps — it is the generic
|
| 272 |
+
"you are a helpful assistant" turn that degrades output (see
|
| 273 |
+
[Generation settings](#generation-settings)). Put the required output format in it.
|
| 274 |
+
- **English prompt text with Uzbek labels** is what was measured. Uzbek prompt wording
|
| 275 |
+
also works; if you change the wording, re-measure — label boundaries (especially
|
| 276 |
+
`Siyosat` vs `Xorijiy Yangiliklar`, and `Oila va Jamiyat`, the weakest class at 0.4273)
|
| 277 |
+
are sensitive to how the categories are described.
|
| 278 |
+
- **Do not batch-compare greedy runs at different batch sizes.** Left-padding shifts the
|
| 279 |
+
numerics; identical prompts matched in only 24 of 32 cases between batch 1 and batch 12.
|
| 280 |
+
|
| 281 |
### Serving
|
| 282 |
|
| 283 |
**vLLM and SGLang cannot load this model.** They reimplement each architecture
|