Model card: drop Training + Data composition, condense Limitations

README.md

@@ -123,48 +123,6 @@ serve).
 
 ---
 
-## Training
-
-MustaqiLLM is the instruction-following stage on top of an in-house 5.17 B Uzbek
-pretrained model. The pretrained checkpoint completed a single pass over its corpus;
-this stage is supervised fine-tuning only — no continued pretraining was performed.
-
-| | |
-|---|---|
-| Method | Full-parameter SFT (no LoRA) |
-| Data | 190,939 instruction/chat examples |
-| Epochs | 3 (17,701 steps); released checkpoint is from epoch 2.75 |
-| Effective batch | 32 (micro-batch 8 × grad-accum 4) |
-| Sequence length | 2048 |
-| Optimizer | AdamW fused, β₁ 0.9, β₂ 0.95, weight decay 0.0 |
-| LR schedule | 1e-5, cosine, 3% warmup, grad-norm clip 1.0 |
-| Precision | bf16 mixed precision, gradient checkpointing |
-| Hardware | 1 × NVIDIA RTX PRO 6000 Blackwell (96 GB), ~10 h |
-
-### Data composition
-
-| slice | rows | share |
-|---|---:|---:|
-| Uzbek instruction/chat backbone (curated + filtered) | 169,919 | 89.0% |
-| Uzbek **Cyrillic** chat (transliterated) | 12,000 | 6.3% |
-| Russian-instructed translation | 5,000 | 2.6% |
-| Latin ↔ Cyrillic script conversion | 3,000 | 1.6% |
-| Cyrillic identity/social | 1,020 | 0.5% |
-
-The backbone mixes general Uzbek assistant data, benchmark-format task data
-(MCQ, classification, spelling), English↔Uzbek translation pairs, and an English
-retention slice. Third-person rubric-grading text was filtered out of the backbone
-before training.
-
-The Cyrillic and Russian slices exist because the pretrained model reads and writes Uzbek
-Cyrillic *better* than Latin (bits-per-byte 0.2288 vs 0.3868) yet had almost no Cyrillic
-chat behaviour attached to it, and because Russian-language instructions were nearly
-absent. Checkpoint selection was done by running the full benchmark suite on all 12
-saved checkpoints, not by held-out loss — held-out loss was flat (1.784–1.796) across
-the last two epochs while benchmark scores were still moving.
-
----
-
 ## Evaluation
 
 Full public benchmark suite, greedy decoding, `transformers` backend, seed 42, complete
@@ -221,41 +179,13 @@ Random baselines: 0.25 for the 4-way MCQ tasks, 0.10 for news, 0.50 for sentimen
 
 ## Limitations
 
-**Multiple-choice knowledge tasks perform at chance.** uzlib, MMLU-Uz and MMLU-English
-all sit within noise of their 0.25 random baseline, across roughly 30,000 questions.
-Invalid rates near zero mean the model answers in the correct format every time and is
-still wrong — this is missing knowledge, not broken parsing. The underlying pretrained
-model completed a single pretraining epoch, and supervised fine-tuning cannot add facts
-that were never learned. **Do not use this model for factual question answering, exams,
-or retrieval-free knowledge tasks.** TUMLU-Uzbek at 0.3286 is the only MCQ result above
-chance, and its 700-item sample gives it a ±3.5% confidence interval.
-
-**Uzbek → English translation is weak.** BLEU 1.83 with a 1.229 length ratio and 12.5% unigram precision means the model
-over-generates English that mostly does not match the reference. English → Uzbek is
-usable (COMET 0.7397) but not competitive with dedicated translation systems.
-
-**Script conversion does not work despite being trained for it.** Asked to transliterate
-Latin Uzbek to Cyrillic, the model frequently returns the input unchanged. The 3,000-row
-slice was too small.
-
-**The Cyrillic slice was machine-transliterated, and its artifacts are visible in
-output.** Loanwords and brand names inside Cyrillic text can come out mangled
-(e.g. `Facebook` → `Факебоок`), and occasional single Cyrillic characters leak into
-Latin words. Cyrillic *chat* is coherent and does not degenerate, but Cyrillic
-*orthography* is less reliable than Latin.
-
-**Self-identification.** The identity training data predates the current name, so asked
-who it is, the model answers "NeuronAI 5B" rather than "MustaqiLLM".
-
-**News classification is uneven.** The "Oila va Jamiyat" (Family & Society) class scores
-0.4273 across 14,012 items — a semantically diffuse catch-all the model handles poorly,
-against 0.8743 for the lexically distinctive Sport class.
-
-**Safety.** No safety alignment, RLHF, or red-teaming was performed. The model has no
-refusal training beyond what the instruction data incidentally contains. It can produce
-incorrect, biased, or unsafe content, and — given the benchmark results above — will
-state false facts fluently and confidently. Evaluate it for your own use case before
-deploying it anywhere user-facing.
+- **MCQ knowledge tasks are at chance.** uzlib, MMLU-Uz and MMLU-English all sit within noise of the 0.25 baseline over ~30,000 questions, with near-zero invalid rates — correct format, wrong answer. This is missing knowledge, not parsing. Do not use it for factual QA, exams, or retrieval-free knowledge tasks. TUMLU-Uzbek (0.3286) is the only MCQ result above chance, on a 700-item sample (±3.5%).
+- **Uzbek → English translation is weak** (BLEU 1.83, length ratio 1.229): it over-generates. English → Uzbek is usable (COMET 0.7397) but below dedicated MT systems.
+- **Script conversion does not work** despite being trained for it — Latin→Cyrillic requests often return the input unchanged.
+- **Cyrillic artifacts.** The Cyrillic data was machine-transliterated; loanwords and brand names can be mangled (`Facebook` → `Факебоок`) and stray Cyrillic characters leak into Latin words. Cyrillic chat is coherent, but its orthography is less reliable than Latin.
+- **Self-identification.** Identity data predates the current name, so the model calls itself "NeuronAI 5B".
+- **Uneven news classification:** 0.4273 on the diffuse "Oila va Jamiyat" class vs 0.8743 on Sport.
+- **Safety.** No safety alignment, RLHF, or red-teaming; no refusal training beyond what the instruction data incidentally contains. It can produce incorrect, biased, or unsafe content and will state false facts fluently. Evaluate before any user-facing deployment.
 
 ---