Match NeuronAI-4B card format with 2B evidence and Apache license

.gitattributes

@@ -35,3 +35,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
 assets/suite_by_task.png filter=lfs diff=lfs merge=lfs -text
+assets/checkpoint_selection.png filter=lfs diff=lfs merge=lfs -text
+assets/tokenizer_fertility.png filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,19 +1,23 @@
 ---
 language:
-- uz
-- en
+  - uz
+  - en
 license: apache-2.0
 library_name: transformers
 pipeline_tag: text-generation
 base_model: Qwen/Qwen3.5-2B-Base
 tags:
-- qwen3.5
-- uzbek
-- text-generation
-- conversational
-- axolotl
-- peft
-- lora
+  - qwen3.5
+  - uzbek
+  - conversational
+  - translation
+  - text-generation-inference
+datasets:
+  - HuggingFaceFW/fineweb-2
+  - tahrirchi/uz-books
+  - tahrirchi/uz-crawl
+  - HuggingFaceFW/fineweb-edu
+  - HuggingFaceTB/finemath
 ---
 
 # NeuronAI-2B
@@ -23,7 +27,11 @@ Qwen3.5-2B-Base. It combines an Uzbek tokenizer retrofit, continued pretraining,
 annealing, and assistant-only supervised fine-tuning. The published weights are
 fully merged—no LoRA adapter is needed.
 
-![Strict eight-task benchmark comparison](assets/weighted_score.png)
+![Strict eight-task benchmark comparison](assets/overall_score.png)
+
+> **License:** [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
+> Commercial and non-commercial use are permitted under the license terms. This
+> differs from the NeuronAI-4B release, which is licensed for non-commercial use.
 
 ## Quick start
 
@@ -42,7 +50,7 @@ tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
     dtype=torch.bfloat16,
-    device_map="cuda:0",
+    device_map={"": 0},
 ).eval()
 
 messages = [
@@ -58,7 +66,17 @@ inputs = tokenizer.apply_chat_template(
 ).to(model.device)
 
 with torch.inference_mode():
-    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
+    output = model.generate(
+        **inputs,
+        max_new_tokens=1024,
+        do_sample=True,
+        temperature=0.7,
+        top_p=0.8,
+        top_k=20,
+        min_p=0.0,
+        repetition_penalty=1.0,
+        use_cache=True,
+    )
 
 reply = tokenizer.decode(
     output[0, inputs["input_ids"].shape[1]:],
@@ -67,8 +85,12 @@ reply = tokenizer.decode(
 print(reply)
 ```
 
-The generation metadata already registers `<|im_end|>` and `<|endoftext|>` as
-end-of-sequence tokens.
+This is the recommended quality-oriented preset for general assistant use:
+non-thinking mode with Qwen3.5's instruct sampling settings. Greedy decoding
+can cause repetition and lower response quality; reserve `do_sample=False` for
+deterministic evaluation or classification. The generation metadata already
+registers `<|im_end|>` and `<|endoftext|>` as end-of-sequence tokens. Keep the
+combined prompt and output within the validated 4,096-token serving limit.
 
 ### Serve with vLLM
 
@@ -77,7 +99,11 @@ pip install -U vllm
 vllm serve NeuronUz/NeuronAI-2B \
   --dtype bfloat16 \
   --max-model-len 4096 \
+  --tensor-parallel-size 1 \
+  --generation-config vllm \
+  --default-chat-template-kwargs '{"enable_thinking":false}' \
   --language-model-only \
+  --enable-prefix-caching \
   --mamba-block-size 16 \
   --mamba-cache-mode align
 ```
@@ -85,86 +111,169 @@ vllm serve NeuronUz/NeuronAI-2B \
 ```bash
 curl http://localhost:8000/v1/chat/completions \
   -H "Content-Type: application/json" \
-  -d '{"model":"NeuronUz/NeuronAI-2B","messages":[{"role":"user","content":"O‘zbekiston haqida uchta fakt ayting."}]}'
+  -d '{
+    "model": "NeuronUz/NeuronAI-2B",
+    "messages": [
+      {"role": "user", "content": "O‘zbekiston haqida uchta fakt ayting."}
+    ],
+    "max_tokens": 1024,
+    "temperature": 0.7,
+    "top_p": 0.8,
+    "top_k": 20,
+    "min_p": 0.0,
+    "presence_penalty": 1.5,
+    "repetition_penalty": 1.0,
+    "chat_template_kwargs": {"enable_thinking": false}
+  }'
 ```
 
 ## Benchmarks
 
-The results below are recomputed from archived full-suite runs using the same
-eight tasks, COMET-primary scoring, and normalized task weights. Classification
-and multiple-choice tasks use accuracy; FLORES+ translation uses COMET. All
-format-scored NeuronAI-2B tasks had a 0% invalid-output rate.
+All five model result sets below cover the same full eight-task suite.
+Classification and multiple-choice tasks use accuracy; FLORES+ translation
+uses COMET. The weighted score is normalized by the 0.95 sum of the published
+task weights. All eight NeuronAI-2B tasks completed and passed the
+invalid-output gate.
+
+![Per-task comparison](assets/tasks_comparison.png)
+
+| Benchmark | Metric | Weight | **NeuronAI-2B** | Qwen3.5-2B | alloma-8B | alloma-3B | alloma-1B |
+| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| UzLiB | accuracy | 0.20 | **49.60%** | 28.69% | 42.40% | 32.08% | 23.32% |
+| TUMLU-Uzbek | accuracy | 0.20 | **32.57%** | 31.29% | 20.71% | 27.71% | 22.00% |
+| FLORES+ en→uz | COMET | 0.15 | 0.8762 | 0.7010 | **0.8779** | 0.8673 | 0.7383 |
+| Uzbek news | accuracy | 0.10 | **78.55%** | 36.75% | 57.77% | 13.60% | 25.41% |
+| MMLU English | accuracy | 0.10 | **54.07%** | 52.39% | 53.47% | 38.73% | 21.98% |
+| MMLU Uzbek | accuracy | 0.10 | **46.85%** | 37.10% | 40.04% | 32.74% | 21.11% |
+| FLORES+ uz→en | COMET | 0.05 | 0.8535 | 0.8072 | **0.8713** | 0.7954 | 0.7636 |
+| Uzbek sentiment | accuracy | 0.05 | **95.50%** | 76.87% | 79.94% | 38.85% | 79.54% |
+| **Normalized weighted score** | | 1.00 | **0.5954** | 0.4528 | 0.5187 | 0.4147 | 0.3661 |
+
+Alloma runs used the `APST` apostrophe preprocessing required by their model
+cards; NeuronAI and stock Qwen did not. The alloma-8B column combines its full
+model-card-protocol evaluation with separately archived full UzLiB,
+TUMLU-Uzbek, and MMLU-Uzbek runs. Exact source files, scores, and run IDs are
+included in [`benchmark_results.json`](benchmark_results.json).
+
+### Run the benchmarks on your computer
+
+The repository includes a portable Alloma-style benchmark runner. It covers
+FLORES+ (both directions), Uzbek sentiment, Uzbek news, MMLU English, MMLU Uzbek,
+and TUMLU-Uzbek.
 
-![Per-task comparison](assets/suite_by_task.png)
+```bash
+pip install -r https://huggingface.co/NeuronUz/NeuronAI-2B/resolve/main/benchmark-requirements.txt
+wget https://huggingface.co/NeuronUz/NeuronAI-2B/resolve/main/benchmark.py
+python benchmark.py --limit 200 --output quick-results.json
+```
 
-| Benchmark | Metric | Weight | **NeuronAI-2B** | Qwen3.5-2B | alloma-3B | alloma-1B |
-| --- | --- | ---: | ---: | ---: | ---: | ---: |
-| UzLiB | accuracy | 0.20 | **49.60%** | 28.69% | 32.08% | 23.32% |
-| TUMLU-Uzbek | accuracy | 0.20 | **32.57%** | 31.29% | 27.71% | 22.00% |
-| FLORES+ en→uz | COMET | 0.15 | **0.8762** | 0.7010 | 0.8673 | 0.7383 |
-| Uzbek news | accuracy | 0.10 | **78.55%** | 36.75% | 13.60% | 25.41% |
-| MMLU English | accuracy | 0.10 | **54.07%** | 52.39% | 38.73% | 21.98% |
-| MMLU Uzbek | accuracy | 0.10 | **46.85%** | 37.10% | 32.74% | 21.11% |
-| FLORES+ uz→en | COMET | 0.05 | **0.8535** | 0.8072 | 0.7954 | 0.7636 |
-| Uzbek sentiment | accuracy | 0.05 | **95.50%** | 76.87% | 38.85% | 79.54% |
-| **Normalized weighted score** | | 1.00 | **0.5954** | 0.4528 | 0.4147 | 0.3661 |
-
-The archived runs came from three suite revisions, but every run computed COMET,
-allowing the table to use one scoring method. Alloma runs used the `APST`
-apostrophe preprocessing required by their model cards.
+The quick command uses the same seed on 200 examples per dataset. Run all public
+examples and add COMET with:
 
-## Uzbek tokenizer efficiency
+```bash
+pip install unbabel-comet
+python benchmark.py --limit 0 --comet --output full-results.json
+```
 
-The tokenizer is an in-place retrofit rather than a vocabulary extension:
-Uzbek merges replace low-value slots while the 248,044-token vocabulary and
-parameter count remain unchanged.
+Run one task when you only need a short check:
 
-![Uzbek tokenizer fertility](assets/fertility_uzcrawl.png)
+```bash
+python benchmark.py --tasks mmlu-uz --limit 200 --output mmlu-uz.json
+python benchmark.py --tasks flores --limit 200 --output flores.json
+```
 
-| Tokenizer | Vocabulary | Tokens per word |
-| --- | ---: | ---: |
-| NeuronAI-2B | 248,044 | **2.407** |
-| Qwen3.5-2B | 248,044 | 3.426 |
+`--limit 0` means the full dataset. Only full runs are comparable with the table
+above; 200-example quick runs are sanity checks. COMET downloads the
+`Unbabel/wmt22-comet-da` evaluator and needs additional disk/RAM.
 
-This measurement used 20,000 documents (4.8M whitespace words) sampled with
-seed 42 from `uz-crawl`. The retrofit uses about 30% fewer tokens on this
-in-domain Uzbek text. Fertility measures tokenization efficiency, not model
-quality or measured inference speed.
+## Uzbek tokenizer efficiency
 
-## Model lineage and training
+The tokenizer is an in-place, primarily **Latin-script Uzbek** retrofit rather
+than a vocabulary extension. The initial 20,000-document figure was measured on
+training-source `uz-crawl`, so we replaced it with a larger corpus-stratified
+test: 118,832 held-out-source documents plus a separate 100,000-document
+training-source control. Documents were selected with deterministic SHA-256
+bottom-k sampling (seed `20260825`), exact duplicates were excluded from the
+selected sample, tiny texts were filtered, and raw source text was tokenized
+without apostrophe normalization.
 
-1. `Qwen/Qwen3.5-2B-Base`
-2. Uzbek tokenizer retrofit with embedding re-initialization
-3. Uzbek continued pretraining
-4. Instruction-data annealing
-5. Assistant-only SFT with LoRA r64
-6. LoRA merged into the model weights
+![Uzbek tokenizer fertility](assets/tokenizer_fertility.png)
 
-One epoch of assistant-only LoRA fine-tuning (rank 64, alpha 128) on 152,152
-Uzbek-first examples, using a 2,048-token context and effective batch size 32.
-Training data is not distributed with this model repository.
+| Corpus | Status | Documents | Words | NeuronAI-2B | Qwen3.5-2B | Reduction (95% CI) |
+| --- | --- | ---: | ---: | ---: | ---: | ---: |
+| Community OSCAR Uzbek | Held-out web source | 100,000 | 7,618,770 | **2.0304** | 3.3639 | **39.64%** (39.57–39.71%) |
+| Uzbek legal corpus | Held-out legal source/domain | 18,832 | 2,534,566 | **2.3747** | 2.9705 | **20.06%** (19.55–20.57%) |
+| uz-crawl | Training-source control | 100,000 | 20,825,680 | **2.3206** | 3.3224 | **30.15%** (30.02–30.30%) |
+
+Across the two held-out sources combined, the tokenizer uses **35.19%
+fewer tokens overall** and **40.90% fewer tokens on Latin-dominant text**,
+matching its intended Latin-Uzbek focus.
+
+The paired intervals use 5,000 bootstrap replicates over 1,000 deterministic
+document buckets. OSCAR may still have incidental overlap with other public web
+corpora and was previously checked in a post-hoc weak-token coverage analysis,
+but it contributed no tokenizer-training rows. The legal corpus does not appear
+in the tokenizer or training source manifests and is the cleanest
+source-and-domain holdout in this test. Full results and script/length
+breakdowns: [`fertility_large_20260825.json`](fertility_large_20260825.json) and
+[`fertility_large_20260825.md`](fertility_large_20260825.md).
+
+Fertility measures tokenization efficiency—not model quality or measured
+decoding speed. The evaluated 2B and 4B custom tokenizer files are byte-identical,
+as are their evaluated stock-base tokenizer files; SHA-256 fingerprints are
+recorded in the JSON result.
+
+## Training
+
+| Item | Value |
+| --- | --- |
+| Parameters | 1,881,825,088 (1.882B) |
+| Prepared train examples | 151,968 (152,152 source rows) |
+| Prepared grouped dev examples | 1,535 (1,537 source rows) |
+| Train/dev prompt-group overlap | 0 |
+| Sequence length / packing | 2,048 / disabled |
+| Training duration / seed | 1 epoch / 42 |
+| Batch size | 16 micro × 2 accumulation × 1 GPU = 32 effective |
+| Optimizer | Fused AdamW; betas 0.9/0.95; weight decay 0.01; gradient clipping 1.0 |
+| Learning-rate schedule | Peak 1e-4; cosine decay; 142 warmup steps (2.99%) |
+| LoRA | rank 64, alpha 128, dropout 0.05; 12 projection types; 67,276,800 trainable parameters |
+| Loss | Fused causal-LM cross-entropy on assistant-response tokens; prompt tokens masked |
+| Precision | bf16 training with TF32; merged embeddings and normalization tensors retained in fp32 |
+
+The mixture is Uzbek-first and includes general assistant conversations,
+translation, Uzbek language and literature, spelling, classification, math,
+and English-retention examples. Training data is not distributed with this
+model repository.
 
 ## Intended use
 
-Good fits include Uzbek research, education, prototyping, translation
-experiments, writing assistance, retrieval-augmented generation, and
-local/offline applications.
+Good fits include Uzbek research, education, commercial and non-commercial
+prototyping, translation experiments, writing assistance, retrieval-augmented
+generation, and local/offline applications. Users remain responsible for
+validating the model for their application and complying with the Apache 2.0
+license and applicable law.
 
 ## Limitations
 
-- TUMLU-Uzbek remains close to the 0.25 random baseline and should not be
-  treated as solved.
+- This is a public-suite-selected checkpoint. The benchmark results are useful
+  for reproducibility and relative comparison, but they are not a locked,
+  independent estimate of real-world generalization.
+- LoRA rank, learning rate, batch size, and dropout were not exhaustively swept;
+  the table reports the released run, not globally optimal hyperparameters.
+- TUMLU-Uzbek is the weakest reported Uzbek task and should not be treated as
+  solved at 32.57% accuracy.
 - The model can hallucinate, repeat biases in its data, or produce unsafe or
   outdated content. It has not been comprehensively safety-evaluated.
 - Do not rely on it without expert review for medical, legal, financial, public
   safety, or other high-stakes decisions.
 - SFT used sequences up to 2,048 tokens; serving at longer inherited context
-  lengths has not been validated here.
-- Use a recent Transformers or vLLM release with Qwen3.5 support. Keep the
-  hybrid model on one device unless multi-device Gated-DeltaNet inference has
-  been validated in your serving stack.
+  lengths has not been validated here. The published inference examples use
+  4,096 tokens.
 
 ## License
 
 NeuronAI-2B is released under the
-[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
+[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Commercial
+and non-commercial use, modification, and distribution are permitted subject
+to its terms. This summary does not replace the license text; see
+[`LICENSE`](LICENSE).

benchmark-requirements.txt

@@ -0,0 +1,9 @@
+torch
+transformers>=5.1
+accelerate
+datasets
+sacrebleu
+vllm
+
+# Optional, only for `python benchmark.py --comet`:
+# unbabel-comet

benchmark.py

@@ -0,0 +1,355 @@
+#!/usr/bin/env python3
+"""Portable NeuronAI-2B / Alloma-style Uzbek benchmark runner.
+
+Examples:
+  python benchmark.py --limit 200 --output quick-results.json
+  python benchmark.py --limit 0 --comet --output full-results.json
+
+`--limit 0` evaluates every example. The default 200-example run is a quick,
+deterministically sampled sanity check and must not be compared with the full
+scores in the model card.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+from datetime import datetime, timezone
+from pathlib import Path
+
+import torch
+from datasets import concatenate_datasets, load_dataset
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+
+MODEL_ID = "NeuronUz/NeuronAI-2B"
+LETTERS = "ABCD"
+
+TRANSLATION_PROMPTS = {
+    "uz-en": (
+        "Translate each Uzbek sentence into English.\n\n"
+        "1991-yilning 1-sentabrida O'zbekiston mustaqilligini e'lon qildi.\n"
+        "-> On 1 September 1991, Uzbekistan declared its independence.\n\n"
+        "Tadqiqotchilar yangi usul samaradorligi 47 foizga oshganini aniqladilar.\n"
+        "-> Researchers found that the new method improved efficiency by 47 percent.\n\n"
+        "{text}\n->"
+    ),
+    "en-uz": "Translate into Uzbek:\n\n{text}",
+}
+
+SENTIMENT_PROMPT = (
+    "Given the following Uzbek text, determine the sentiment as either "
+    "'Positive' or 'Negative'. Respond with only one label.\n\nText: {text}\n\nLabel:"
+)
+
+NEWS_PROMPT = """Classify the given Uzbek news article into one category. Respond with only the category number.
+
+0 - Siyosat
+1 - Iqtisodiyot
+2 - Texnologiya
+3 - Sport
+4 - Madaniyat
+5 - Salomatlik
+6 - Oila va Jamiyat
+7 - Ta'lim
+8 - Ekologiya
+9 - Xorijiy Yangiliklar
+
+Article: {text}
+
+Answer:"""
+
+MCQ_SUFFIX = {
+    "uz": "Variantlarni diqqat bilan solishtiring. Javobni A, B, C yoki D harfi bilan boshlang.",
+    "en": "Compare the options carefully. Start with the answer letter A, B, C, or D.",
+}
+
+MCQ_TASKS = {
+    "mmlu-en": ("cais/mmlu", "all", "test", "en"),
+    "mmlu-uz": ("murodbek/MMLU-uz", "default", "test", "uz"),
+    "tumlu": ("jafarisbarov/TUMLU-mini", "uzbek", "test", "uz"),
+}
+
+
+def choose_rows(dataset, limit: int, seed: int):
+    if limit and len(dataset) > limit:
+        return dataset.shuffle(seed=seed).select(range(limit))
+    return dataset
+
+
+def strip_thinking(text: str) -> str:
+    return re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL).strip()
+
+
+class Generator:
+    def __init__(self, args: argparse.Namespace):
+        self.backend = args.backend
+        self.tokenizer = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
+        self.tokenizer.padding_side = "left"
+        if self.tokenizer.pad_token_id is None:
+            self.tokenizer.pad_token_id = self.tokenizer.eos_token_id
+
+        if args.backend == "vllm":
+            from vllm import LLM, SamplingParams
+
+            self.sampling_cls = SamplingParams
+            self.model = LLM(
+                model=args.model,
+                dtype=args.dtype,
+                trust_remote_code=True,
+                gpu_memory_utilization=args.gpu_memory_utilization,
+                max_model_len=args.max_model_len,
+                language_model_only=True,
+                mamba_block_size=16,
+                mamba_cache_mode="align",
+            )
+        else:
+            dtype = torch.bfloat16 if args.dtype == "bfloat16" else torch.float16
+            self.model = AutoModelForCausalLM.from_pretrained(
+                args.model,
+                dtype=dtype,
+                device_map="auto",
+                trust_remote_code=True,
+            ).eval()
+
+    def render(self, prompt: str) -> str:
+        messages = [{"role": "user", "content": prompt}]
+        try:
+            return self.tokenizer.apply_chat_template(
+                messages,
+                tokenize=False,
+                add_generation_prompt=True,
+                enable_thinking=False,
+            )
+        except TypeError:
+            return self.tokenizer.apply_chat_template(
+                messages, tokenize=False, add_generation_prompt=True
+            )
+
+    def generate(self, prompts: list[str], max_new_tokens: int, batch_size: int) -> list[str]:
+        rendered = [self.render(prompt) for prompt in prompts]
+        if self.backend == "vllm":
+            params = self.sampling_cls(temperature=0.0, max_tokens=max_new_tokens)
+            outputs = self.model.generate(rendered, params)
+            return [strip_thinking(item.outputs[0].text) for item in outputs]
+
+        results: list[str] = []
+        for start in range(0, len(rendered), batch_size):
+            batch = rendered[start : start + batch_size]
+            encoded = self.tokenizer(
+                batch,
+                return_tensors="pt",
+                padding=True,
+                truncation=True,
+                max_length=4096,
+            ).to(self.model.device)
+            prompt_width = encoded["input_ids"].shape[1]
+            with torch.inference_mode():
+                output = self.model.generate(
+                    **encoded,
+                    max_new_tokens=max_new_tokens,
+                    do_sample=False,
+                )
+            results.extend(
+                strip_thinking(text)
+                for text in self.tokenizer.batch_decode(
+                    output[:, prompt_width:], skip_special_tokens=True
+                )
+            )
+        return results
+
+
+def load_flores(direction: str, limit: int, seed: int) -> list[dict[str, str]]:
+    langs = {"uz-en": ("uzn_Latn", "eng_Latn"), "en-uz": ("eng_Latn", "uzn_Latn")}
+    src_lang, ref_lang = langs[direction]
+    src = concatenate_datasets([
+        load_dataset("openlanguagedata/flores_plus", src_lang, split="dev"),
+        load_dataset("openlanguagedata/flores_plus", src_lang, split="devtest"),
+    ])
+    ref = concatenate_datasets([
+        load_dataset("openlanguagedata/flores_plus", ref_lang, split="dev"),
+        load_dataset("openlanguagedata/flores_plus", ref_lang, split="devtest"),
+    ])
+    pairs = [
+        {"src": src[index]["text"].strip(), "ref": ref[index]["text"].strip()}
+        for index in range(min(len(src), len(ref)))
+    ]
+    if limit and len(pairs) > limit:
+        import random
+
+        random.Random(seed).shuffle(pairs)
+        pairs = pairs[:limit]
+    return pairs
+
+
+def score_comet(sources: list[str], hypotheses: list[str], references: list[str]) -> float:
+    from comet import download_model, load_from_checkpoint
+
+    checkpoint = download_model("Unbabel/wmt22-comet-da")
+    model = load_from_checkpoint(checkpoint)
+    rows = [
+        {"src": src, "mt": hypothesis, "ref": reference}
+        for src, hypothesis, reference in zip(sources, hypotheses, references, strict=True)
+    ]
+    return float(model.predict(rows, batch_size=8, gpus=1 if torch.cuda.is_available() else 0).system_score)
+
+
+def evaluate_flores(generator: Generator, args: argparse.Namespace) -> dict:
+    import sacrebleu
+
+    results = {}
+    for direction in ("uz-en", "en-uz"):
+        pairs = load_flores(direction, args.limit, args.seed)
+        prompts = [TRANSLATION_PROMPTS[direction].format(text=row["src"]) for row in pairs]
+        hypotheses = generator.generate(prompts, max_new_tokens=160, batch_size=args.batch_size)
+        references = [row["ref"] for row in pairs]
+        sources = [row["src"] for row in pairs]
+        row = {
+            "total": len(pairs),
+            "bleu": float(sacrebleu.corpus_bleu(hypotheses, [references]).score),
+            "samples": [
+                {"source": src, "prediction": hyp, "reference": ref}
+                for src, hyp, ref in zip(sources[:3], hypotheses[:3], references[:3])
+            ],
+        }
+        if args.comet:
+            row["comet"] = score_comet(sources, hypotheses, references)
+        results[direction] = row
+        print(f"FLORES+ {direction}: BLEU={row['bleu']:.2f}" + (f", COMET={row['comet']:.4f}" if args.comet else ""))
+    return results
+
+
+def label_to_int(raw, names: list[str]) -> int | None:
+    if isinstance(raw, int) and 0 <= raw < len(names):
+        return raw
+    cleaned = str(raw).strip().casefold().replace("’", "'")
+    for index, name in enumerate(names):
+        if cleaned == name.casefold():
+            return index
+    return None
+
+
+def evaluate_classification(generator: Generator, args: argparse.Namespace, task: str) -> dict:
+    if task == "sentiment":
+        dataset = load_dataset("behbudiy/uzbek-sentiment-analysis", split="train")
+        names = ["Negative", "Positive"]
+        rows = [
+            {"text": row["text"], "gold": label_to_int(row["label"], names)}
+            for row in choose_rows(dataset, args.limit, args.seed)
+        ]
+        prompt_template = SENTIMENT_PROMPT
+        parser = lambda text: 1 if text.casefold().startswith("positive") else (0 if text.casefold().startswith("negative") else None)
+    else:
+        dataset = load_dataset("risqaliyevds/uzbek-zero-shot-classification", split="train")
+        names = ["Siyosat", "Iqtisodiyot", "Texnologiya", "Sport", "Madaniyat",
+                 "Salomatlik", "Oila va Jamiyat", "Ta'lim", "Ekologiya", "Xorijiy Yangiliklar"]
+        rows = [
+            {"text": row["text"], "gold": label_to_int(row["class"], names)}
+            for row in choose_rows(dataset, args.limit, args.seed)
+        ]
+        prompt_template = NEWS_PROMPT
+        parser = lambda text: int(match.group()) if (match := re.search(r"\d", text)) else None
+
+    rows = [row for row in rows if row["gold"] is not None]
+    prompts = [prompt_template.format(text=row["text"][: args.max_text_chars]) for row in rows]
+    outputs = generator.generate(prompts, max_new_tokens=8, batch_size=args.batch_size)
+    predictions = [parser(output.strip()) for output in outputs]
+    correct = sum(prediction == row["gold"] for prediction, row in zip(predictions, rows, strict=True))
+    invalid = sum(prediction is None for prediction in predictions)
+    result = {
+        "accuracy": correct / len(rows),
+        "correct": correct,
+        "total": len(rows),
+        "invalid_rate": invalid / len(rows),
+    }
+    print(f"{task}: accuracy={result['accuracy']:.2%} ({correct}/{len(rows)}), invalid={invalid}")
+    return result
+
+
+def answer_letter(raw) -> str | None:
+    if isinstance(raw, int) and 0 <= raw < 4:
+        return LETTERS[raw]
+    cleaned = str(raw).strip().upper()
+    return cleaned[0] if cleaned and cleaned[0] in LETTERS else None
+
+
+def evaluate_mcq(generator: Generator, args: argparse.Namespace, task: str) -> dict:
+    dataset_name, config, split, language = MCQ_TASKS[task]
+    dataset = choose_rows(load_dataset(dataset_name, config, split=split), args.limit, args.seed)
+    rows = []
+    for row in dataset:
+        choices = row.get("choices") or [row.get(f"option_{letter.lower()}") for letter in LETTERS]
+        choices = [str(choice) for choice in choices if choice is not None]
+        gold = answer_letter(row.get("answer"))
+        if row.get("question") and len(choices) >= 4 and gold:
+            rows.append({"question": row["question"], "choices": choices[:4], "gold": gold})
+    prompts = []
+    for row in rows:
+        choices = "\n".join(f"{letter}) {choice}" for letter, choice in zip(LETTERS, row["choices"]))
+        prompts.append(f"{row['question']}\n\n{choices}\n\n{MCQ_SUFFIX[language]}")
+    outputs = generator.generate(prompts, max_new_tokens=12, batch_size=args.batch_size)
+    predictions = []
+    for output in outputs:
+        match = re.search(r"[ABCD]", output.upper())
+        predictions.append(match.group() if match else None)
+    correct = sum(prediction == row["gold"] for prediction, row in zip(predictions, rows, strict=True))
+    invalid = sum(prediction is None for prediction in predictions)
+    result = {
+        "accuracy": correct / len(rows),
+        "correct": correct,
+        "total": len(rows),
+        "invalid_rate": invalid / len(rows),
+    }
+    print(f"{task}: accuracy={result['accuracy']:.2%} ({correct}/{len(rows)}), invalid={invalid}")
+    return result
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--model", default=MODEL_ID)
+    parser.add_argument("--tasks", default="flores,sentiment,news,mmlu-en,mmlu-uz,tumlu")
+    parser.add_argument("--backend", choices=["vllm", "transformers"], default="vllm")
+    parser.add_argument("--limit", type=int, default=200, help="Examples per dataset; 0 means full dataset.")
+    parser.add_argument("--batch-size", type=int, default=16)
+    parser.add_argument("--max-text-chars", type=int, default=4000)
+    parser.add_argument("--max-model-len", type=int, default=4096)
+    parser.add_argument("--gpu-memory-utilization", type=float, default=0.85)
+    parser.add_argument("--dtype", choices=["bfloat16", "float16"], default="bfloat16")
+    parser.add_argument("--seed", type=int, default=42)
+    parser.add_argument("--comet", action="store_true", help="Download WMT22-COMET-DA and score FLORES+.")
+    parser.add_argument("--output", type=Path, default=Path("neuronai-2b-benchmark.json"))
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    if args.limit < 0:
+        raise ValueError("--limit must be 0 or greater")
+    tasks = {task.strip() for task in args.tasks.split(",") if task.strip()}
+    unknown = tasks - {"flores", "sentiment", "news", *MCQ_TASKS}
+    if unknown:
+        raise ValueError(f"Unknown tasks: {sorted(unknown)}")
+    generator = Generator(args)
+    results = {
+        "model": args.model,
+        "timestamp": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        "limit_per_dataset": args.limit,
+        "seed": args.seed,
+        "backend": args.backend,
+        "results": {},
+    }
+    if "flores" in tasks:
+        results["results"]["flores"] = evaluate_flores(generator, args)
+    for task in ("sentiment", "news"):
+        if task in tasks:
+            results["results"][task] = evaluate_classification(generator, args, task)
+    for task in MCQ_TASKS:
+        if task in tasks:
+            results["results"][task] = evaluate_mcq(generator, args, task)
+    args.output.write_text(json.dumps(results, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
+    print(f"Wrote {args.output}")
+
+
+if __name__ == "__main__":
+    main()

benchmark_results.json

@@ -0,0 +1,264 @@
+{
+  "schema_version": 1,
+  "metric_note": "Accuracy for non-translation tasks; COMET for FLORES+; weighted sum normalized by 0.95.",
+  "weights": {
+    "uzlib_slice_full": 0.2,
+    "tumlu_uzbek_full": 0.2,
+    "flores_en_uz_full": 0.15,
+    "news_full": 0.1,
+    "mmlu_english_full": 0.1,
+    "mmlu_uz_full": 0.1,
+    "flores_uz_en_full": 0.05,
+    "sentiment_binary_full": 0.05
+  },
+  "labels": {
+    "uzlib_slice_full": "UzLiB",
+    "tumlu_uzbek_full": "TUMLU-Uzbek",
+    "flores_en_uz_full": "FLORES+ en to uz",
+    "news_full": "News",
+    "mmlu_english_full": "MMLU English",
+    "mmlu_uz_full": "MMLU Uzbek",
+    "flores_uz_en_full": "FLORES+ uz to en",
+    "sentiment_binary_full": "Sentiment"
+  },
+  "models": {
+    "NeuronAI-2B": {
+      "run_id": "qwen35_2b_lora_r64_grouped_identity_ckpt4749_public_v2",
+      "suite_id": "public_uzbek_full_comet_vllm_v2",
+      "tasks": {
+        "uzlib_slice_full": 0.4959699086512628,
+        "tumlu_uzbek_full": 0.32571428571428573,
+        "flores_en_uz_full": 0.8761537177251785,
+        "news_full": 0.7854903578426318,
+        "mmlu_english_full": 0.5406637231163652,
+        "mmlu_uz_full": 0.4685230024213075,
+        "flores_uz_en_full": 0.8534744450736248,
+        "sentiment_binary_full": 0.955
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0,
+        "tumlu_uzbek_full": 0.0,
+        "flores_en_uz_full": null,
+        "news_full": 0.0,
+        "mmlu_english_full": 0.0,
+        "mmlu_uz_full": 0.0,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0
+      },
+      "normalized_weighted_score": 0.5954224496037875
+    },
+    "Qwen3.5-2B": {
+      "run_id": "qwen35_2b_instruct_full_20260817_081135",
+      "suite_id": "public_uzbek_full_vllm",
+      "tasks": {
+        "uzlib_slice_full": 0.28694250403009136,
+        "tumlu_uzbek_full": 0.31285714285714283,
+        "flores_en_uz_full": 0.7009904753173863,
+        "news_full": 0.36753635144890173,
+        "mmlu_english_full": 0.5239282153539382,
+        "mmlu_uz_full": 0.3709585529126905,
+        "flores_uz_en_full": 0.8071526469372587,
+        "sentiment_binary_full": 0.7687
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0,
+        "tumlu_uzbek_full": 0.0,
+        "flores_en_uz_full": null,
+        "news_full": 0.0,
+        "mmlu_english_full": 0.0,
+        "mmlu_uz_full": 0.0,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0002
+      },
+      "normalized_weighted_score": 0.45278257367733765
+    },
+    "alloma-8B": {
+      "run_id": "alloma_8b_full_public_composite",
+      "suite_id": "public_uzbek_full_composite_apst",
+      "source_files": [
+        "reports/alloma_card_protocol_full/alloma-8B.json",
+        "reports/eval_suites/external_public/alloma_8b_uzlib_full_20260615/uzlib_full.json",
+        "reports/eval_suites/public_full_comet/alloma_8b_strict_missing_uz_metrics/tumlu_uzbek_full_apst.json",
+        "reports/eval_suites/public_full_comet/alloma_8b_strict_missing_uz_metrics/mmlu_uz_full_apst.json"
+      ],
+      "tasks": {
+        "uzlib_slice_full": 0.4239656098871574,
+        "tumlu_uzbek_full": 0.20714285714285716,
+        "flores_en_uz_full": 0.8779,
+        "news_full": 0.5777,
+        "mmlu_english_full": 0.5347,
+        "mmlu_uz_full": 0.40037031761857284,
+        "flores_uz_en_full": 0.8713,
+        "sentiment_binary_full": 0.7994
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0338527673293928,
+        "tumlu_uzbek_full": 0.25857142857142856,
+        "flores_en_uz_full": null,
+        "news_full": null,
+        "mmlu_english_full": null,
+        "mmlu_uz_full": 0.09051417177040308,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": null
+      },
+      "normalized_weighted_score": 0.5186512896503792
+    },
+    "alloma-3B": {
+      "run_id": "alloma_3B_apst_full_20260815_142714",
+      "suite_id": "public_uzbek_full_vllm_apst",
+      "tasks": {
+        "uzlib_slice_full": 0.32079527135948416,
+        "tumlu_uzbek_full": 0.27714285714285714,
+        "flores_en_uz_full": 0.8673252673692642,
+        "news_full": 0.1359802000618748,
+        "mmlu_english_full": 0.3872667711152257,
+        "mmlu_uz_full": 0.32744623273038026,
+        "flores_uz_en_full": 0.7954316840740269,
+        "sentiment_binary_full": 0.3885
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0010746910263299302,
+        "tumlu_uzbek_full": 0.28714285714285714,
+        "flores_en_uz_full": null,
+        "news_full": 7.218727441476746e-05,
+        "mmlu_english_full": 0.06900726392251816,
+        "mmlu_uz_full": 0.07726819541375872,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.4405
+      },
+      "normalized_weighted_score": 0.41468665305295505
+    },
+    "alloma-1B": {
+      "run_id": "alloma_1B_apst_full_20260811_114037",
+      "suite_id": "public_uzbek_full_vllm_apst",
+      "tasks": {
+        "uzlib_slice_full": 0.23320795271359485,
+        "tumlu_uzbek_full": 0.22,
+        "flores_en_uz_full": 0.7383331350348018,
+        "news_full": 0.2541404558110756,
+        "mmlu_english_full": 0.21976926363765845,
+        "mmlu_uz_full": 0.21108104258652613,
+        "flores_uz_en_full": 0.7636193416348852,
+        "sentiment_binary_full": 0.7954
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.04567436861902203,
+        "tumlu_uzbek_full": 0.36714285714285716,
+        "flores_en_uz_full": null,
+        "news_full": 0.0003506239043002991,
+        "mmlu_english_full": 0.09393248824953711,
+        "mmlu_uz_full": 0.17882068081469876,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0063
+      },
+      "normalized_weighted_score": 0.3661490569296942
+    }
+  },
+  "checkpoints": {
+    "0.25 epoch": {
+      "run_id": "qwen35_2b_lora_r64_grouped_identity_ckpt1188_public_v2",
+      "suite_id": "public_uzbek_full_comet_vllm_v2",
+      "tasks": {
+        "uzlib_slice_full": 0.5432563138097797,
+        "tumlu_uzbek_full": 0.32,
+        "flores_en_uz_full": 0.8569777911726428,
+        "news_full": 0.7521810869341033,
+        "mmlu_english_full": 0.5305512035322604,
+        "mmlu_uz_full": 0.4598347813701752,
+        "flores_uz_en_full": 0.7069662087233046,
+        "sentiment_binary_full": 0.9087
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0,
+        "tumlu_uzbek_full": 0.0,
+        "flores_en_uz_full": null,
+        "news_full": 0.0,
+        "mmlu_english_full": 0.0,
+        "mmlu_uz_full": 0.0,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0
+      },
+      "normalized_weighted_score": 0.5855136305870227
+    },
+    "0.50 epoch": {
+      "run_id": "qwen35_2b_lora_r64_grouped_identity_ckpt2376_public_v2",
+      "suite_id": "public_uzbek_full_comet_vllm_v2",
+      "tasks": {
+        "uzlib_slice_full": 0.4954325631380978,
+        "tumlu_uzbek_full": 0.3057142857142857,
+        "flores_en_uz_full": 0.8641375761503245,
+        "news_full": 0.7519542126430855,
+        "mmlu_english_full": 0.5307648483122063,
+        "mmlu_uz_full": 0.46360917248255235,
+        "flores_uz_en_full": 0.8504324527211662,
+        "sentiment_binary_full": 0.9502
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0,
+        "tumlu_uzbek_full": 0.0,
+        "flores_en_uz_full": null,
+        "news_full": 0.0,
+        "mmlu_english_full": 0.0,
+        "mmlu_uz_full": 0.0,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0
+      },
+      "normalized_weighted_score": 0.5836994233398612
+    },
+    "0.75 epoch": {
+      "run_id": "qwen35_2b_lora_r64_grouped_identity_ckpt3564_public_v2",
+      "suite_id": "public_uzbek_full_comet_vllm_v2",
+      "tasks": {
+        "uzlib_slice_full": 0.4744760881246642,
+        "tumlu_uzbek_full": 0.30857142857142855,
+        "flores_en_uz_full": 0.8769590624563133,
+        "news_full": 0.7998040631123028,
+        "mmlu_english_full": 0.5434411052556616,
+        "mmlu_uz_full": 0.46802449793476714,
+        "flores_uz_en_full": 0.8535260226727719,
+        "sentiment_binary_full": 0.9529
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0,
+        "tumlu_uzbek_full": 0.0,
+        "flores_en_uz_full": null,
+        "news_full": 0.0,
+        "mmlu_english_full": 0.0,
+        "mmlu_uz_full": 0.0,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0
+      },
+      "normalized_weighted_score": 0.5890543478648181
+    },
+    "1.00 epoch": {
+      "run_id": "qwen35_2b_lora_r64_grouped_identity_ckpt4749_public_v2",
+      "suite_id": "public_uzbek_full_comet_vllm_v2",
+      "tasks": {
+        "uzlib_slice_full": 0.4959699086512628,
+        "tumlu_uzbek_full": 0.32571428571428573,
+        "flores_en_uz_full": 0.8761537177251785,
+        "news_full": 0.7854903578426318,
+        "mmlu_english_full": 0.5406637231163652,
+        "mmlu_uz_full": 0.4685230024213075,
+        "flores_uz_en_full": 0.8534744450736248,
+        "sentiment_binary_full": 0.955
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0,
+        "tumlu_uzbek_full": 0.0,
+        "flores_en_uz_full": null,
+        "news_full": 0.0,
+        "mmlu_english_full": 0.0,
+        "mmlu_uz_full": 0.0,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": 0.0
+      },
+      "normalized_weighted_score": 0.5954224496037875
+    }
+  },
+  "tokenizer_fertility": {
+    "status": "superseded",
+    "result": "fertility_large_20260825.json",
+    "note": "Use the corpus-stratified 2026-08-25 evaluation; the old 20k uz-crawl-only result was in-domain."
+  }
+}

fertility_large_20260825.json

@@ -0,0 +1,643 @@
+{
+  "schema_version": 1,
+  "created_at": "2026-08-24T19:25:09+00:00",
+  "seed": 20260825,
+  "sampling": {
+    "method": "deterministic SHA-256 bottom-k, exact duplicates excluded from selected sample",
+    "requested_documents_per_corpus": 100000,
+    "min_chars": 250,
+    "max_chars": 20000,
+    "min_words": 20
+  },
+  "tokenizers": {
+    "new": "out/qwen35_2b_sft_lora_r64_grouped_identity_v1/merges/ckpt4749/merged",
+    "base": "/mnt/raid1/qwen3.5/pipelines/uzbek_tokenizer/artifacts/base_checkpoints/qwen35_2b_base",
+    "result_reuse_note": "The evaluated 2B tokenizer files are byte-identical to the evaluated 4B tokenizer files.",
+    "new_tokenizer_sha256": "e4731e5df2c95dbfd98c45e08feae7c5fcd0588172778ac35c3703921f5529f9",
+    "base_tokenizer_sha256": "fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927",
+    "add_special_tokens": false,
+    "raw_text_no_preprocessing": true
+  },
+  "corpora": {
+    "OSCAR Uzbek": {
+      "domain_status": "held-out web source",
+      "source": "xkas2001/uzbek-language-dataset, community-oscar-uzbek/all_metadata_text.txt",
+      "sampling": {
+        "records_scanned": 4762756,
+        "valid_records": 623128,
+        "sampled_records": 100000,
+        "selected_duplicate_occurrences_skipped": 316611,
+        "records_truncated_before_sampling": 67
+      },
+      "result": {
+        "documents": 100000,
+        "words": 7618770,
+        "characters": 62308307,
+        "new_tokens": 15469208,
+        "base_tokens": 25628881,
+        "new_tokens_per_word": 2.0304075329744826,
+        "base_tokens_per_word": 3.363913203837365,
+        "token_reduction": 0.3964150053995724,
+        "token_reduction_95ci": [
+          0.3956999048249437,
+          0.3971071790721872
+        ],
+        "median_document_tokens_per_word": {
+          "new": 1.9714285714285715,
+          "base": 3.3536585365853657
+        },
+        "document_token_count_comparison": {
+          "new_fewer_fraction": 0.99789,
+          "equal_fraction": 0.00171,
+          "new_more_fraction": 0.0004
+        },
+        "document_length_chars": {
+          "min": 250,
+          "median": 448,
+          "p90": 1105,
+          "p99": 2908,
+          "max": 19999
+        },
+        "by_script": {
+          "latin_dominant": {
+            "documents": 99876,
+            "words": 7609046,
+            "characters": 62243858,
+            "new_tokens": 15444043,
+            "base_tokens": 25601074,
+            "new_tokens_per_word": 2.029695049813078,
+            "base_tokens_per_word": 3.3645576593964606,
+            "token_reduction": 0.39674237885488706,
+            "token_reduction_95ci": [
+              0.3960664169063509,
+              0.3974315537014118
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.9714285714285715,
+              "base": 3.3541666666666665
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9980375665825624,
+              "equal_fraction": 0.001571949217029116,
+              "new_more_fraction": 0.00039048420040850655
+            }
+          },
+          "cyrillic_dominant": {
+            "documents": 121,
+            "words": 8853,
+            "characters": 61058,
+            "new_tokens": 21987,
+            "base_tokens": 24636,
+            "new_tokens_per_word": 2.4835648932565233,
+            "base_tokens_per_word": 2.782785496441884,
+            "token_reduction": 0.10752557233317095,
+            "token_reduction_95ci": [
+              0.06903186700461168,
+              0.1489608189386225
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.220779220779221,
+              "base": 2.5869565217391304
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9008264462809917,
+              "equal_fraction": 0.09917355371900827,
+              "new_more_fraction": 0.0
+            }
+          },
+          "other": {
+            "documents": 3,
+            "words": 871,
+            "characters": 3391,
+            "new_tokens": 3178,
+            "base_tokens": 3171,
+            "new_tokens_per_word": 3.6486796785304247,
+            "base_tokens_per_word": 3.640642939150402,
+            "token_reduction": -0.002207505518763808,
+            "token_reduction_95ci": [
+              -0.06363636363636371,
+              0.0
+            ],
+            "median_document_tokens_per_word": {
+              "new": 3.7318295739348373,
+              "base": 3.7318295739348373
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.0,
+              "equal_fraction": 0.6666666666666666,
+              "new_more_fraction": 0.3333333333333333
+            }
+          }
+        },
+        "by_document_length_quartile": {
+          "q1": {
+            "char_bounds": [
+              250,
+              328
+            ],
+            "documents": 25225,
+            "words": 886299,
+            "characters": 7229107,
+            "new_tokens": 1742025,
+            "base_tokens": 2981550,
+            "new_tokens_per_word": 1.9655048691243022,
+            "base_tokens_per_word": 3.3640453165353903,
+            "token_reduction": 0.41573175026412434,
+            "token_reduction_95ci": [
+              0.41438530829534354,
+              0.4170515905402432
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.9142857142857144,
+              "base": 3.3636363636363638
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9958771060455897,
+              "equal_fraction": 0.004083250743310208,
+              "new_more_fraction": 3.9643211100099105e-05
+            }
+          },
+          "q2": {
+            "char_bounds": [
+              329,
+              448
+            ],
+            "documents": 24838,
+            "words": 1159735,
+            "characters": 9510849,
+            "new_tokens": 2254196,
+            "base_tokens": 3895455,
+            "new_tokens_per_word": 1.943716452465434,
+            "base_tokens_per_word": 3.358918201140778,
+            "token_reduction": 0.42132664861999436,
+            "token_reduction_95ci": [
+              0.4200747806331761,
+              0.42257126807643025
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.9166666666666667,
+              "base": 3.3617021276595747
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9986311297205894,
+              "equal_fraction": 0.001328609388839681,
+              "new_more_fraction": 4.0260890570899426e-05
+            }
+          },
+          "q3": {
+            "char_bounds": [
+              449,
+              694
+            ],
+            "documents": 24973,
+            "words": 1683343,
+            "characters": 13783621,
+            "new_tokens": 3372637,
+            "base_tokens": 5654412,
+            "new_tokens_per_word": 2.0035352272234475,
+            "base_tokens_per_word": 3.359037344141984,
+            "token_reduction": 0.4035388648722449,
+            "token_reduction_95ci": [
+              0.4023960313587086,
+              0.4046209067071377
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.9710144927536233,
+              "base": 3.3466666666666667
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9995194810395227,
+              "equal_fraction": 0.0004004324670644296,
+              "new_more_fraction": 8.008649341288591e-05
+            }
+          },
+          "q4": {
+            "char_bounds": [
+              695,
+              19999
+            ],
+            "documents": 24964,
+            "words": 3889393,
+            "characters": 31784730,
+            "new_tokens": 8100350,
+            "base_tokens": 13097464,
+            "new_tokens_per_word": 2.0826771683910574,
+            "base_tokens_per_word": 3.367482792302038,
+            "token_reduction": 0.3815329440874966,
+            "token_reduction_95ci": [
+              0.38033518553908646,
+              0.38274511916670123
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.05603738317757,
+              "base": 3.347826086956522
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9975564813331197,
+              "equal_fraction": 0.00100144207659029,
+              "new_more_fraction": 0.0014420765902900176
+            }
+          }
+        }
+      }
+    },
+    "Uzbek legal corpus": {
+      "domain_status": "held-out legal source/domain",
+      "source": "sukhrobnurali/uzbek-legal-corpus-v1, articles",
+      "sampling": {
+        "records_scanned": 21021,
+        "valid_records": 18832,
+        "sampled_records": 18832,
+        "selected_duplicate_occurrences_skipped": 0,
+        "records_truncated_before_sampling": 3
+      },
+      "result": {
+        "documents": 18832,
+        "words": 2534566,
+        "characters": 21314804,
+        "new_tokens": 6018932,
+        "base_tokens": 7529043,
+        "new_tokens_per_word": 2.374738712663233,
+        "base_tokens_per_word": 2.970545253112367,
+        "token_reduction": 0.20057144048719078,
+        "token_reduction_95ci": [
+          0.19550471065059247,
+          0.20567097138760268
+        ],
+        "median_document_tokens_per_word": {
+          "new": 1.8397191231160697,
+          "base": 3.3333333333333335
+        },
+        "document_token_count_comparison": {
+          "new_fewer_fraction": 0.385089209855565,
+          "equal_fraction": 0.614910790144435,
+          "new_more_fraction": 0.0
+        },
+        "document_length_chars": {
+          "min": 250,
+          "median": 812,
+          "p90": 2183,
+          "p99": 5637,
+          "max": 19997
+        },
+        "by_script": {
+          "latin_dominant": {
+            "documents": 6354,
+            "words": 858053,
+            "characters": 7499938,
+            "new_tokens": 1410490,
+            "base_tokens": 2916524,
+            "new_tokens_per_word": 1.6438261972162558,
+            "base_tokens_per_word": 3.3990021595402613,
+            "token_reduction": 0.51637977263345,
+            "token_reduction_95ci": [
+              0.5147582025646525,
+              0.5179880323158793
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.6153846153846154,
+              "base": 3.369890856442811
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 1.0,
+              "equal_fraction": 0.0,
+              "new_more_fraction": 0.0
+            }
+          },
+          "cyrillic_dominant": {
+            "documents": 12478,
+            "words": 1676513,
+            "characters": 13814866,
+            "new_tokens": 4608442,
+            "base_tokens": 4612519,
+            "new_tokens_per_word": 2.74882568760278,
+            "base_tokens_per_word": 2.7512575208185086,
+            "token_reduction": 0.0008838987980320789,
+            "token_reduction_95ci": [
+              0.000797593938754379,
+              0.0009744198560997175
+            ],
+            "median_document_tokens_per_word": {
+              "new": 3.1314814814814813,
+              "base": 3.1314814814814813
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.07196666132393012,
+              "equal_fraction": 0.9280333386760699,
+              "new_more_fraction": 0.0
+            }
+          }
+        },
+        "by_document_length_quartile": {
+          "q1": {
+            "char_bounds": [
+              250,
+              506
+            ],
+            "documents": 4715,
+            "words": 213503,
+            "characters": 1769357,
+            "new_tokens": 505325,
+            "base_tokens": 619444,
+            "new_tokens_per_word": 2.366828569153595,
+            "base_tokens_per_word": 2.901336280989026,
+            "token_reduction": 0.1842281142443869,
+            "token_reduction_95ci": [
+              0.17673651196691142,
+              0.19222917798968833
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.868421052631579,
+              "base": 3.2857142857142856
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.34612937433722163,
+              "equal_fraction": 0.6538706256627783,
+              "new_more_fraction": 0.0
+            }
+          },
+          "q2": {
+            "char_bounds": [
+              507,
+              812
+            ],
+            "documents": 4703,
+            "words": 369537,
+            "characters": 3066781,
+            "new_tokens": 873369,
+            "base_tokens": 1074262,
+            "new_tokens_per_word": 2.3634142183326703,
+            "base_tokens_per_word": 2.907048549942225,
+            "token_reduction": 0.18700559081490364,
+            "token_reduction_95ci": [
+              0.17978162099648665,
+              0.19424970760537358
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.8421052631578947,
+              "base": 3.3076923076923075
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.35721879651286415,
+              "equal_fraction": 0.6427812034871359,
+              "new_more_fraction": 0.0
+            }
+          },
+          "q3": {
+            "char_bounds": [
+              813,
+              1347
+            ],
+            "documents": 4711,
+            "words": 594780,
+            "characters": 4943320,
+            "new_tokens": 1407482,
+            "base_tokens": 1750853,
+            "new_tokens_per_word": 2.366390934463163,
+            "base_tokens_per_word": 2.9436985103735833,
+            "token_reduction": 0.19611640725977564,
+            "token_reduction_95ci": [
+              0.18856405345722932,
+              0.20422906152284168
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.8333333333333333,
+              "base": 3.3286713286713288
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.3922734026745914,
+              "equal_fraction": 0.6077265973254086,
+              "new_more_fraction": 0.0
+            }
+          },
+          "q4": {
+            "char_bounds": [
+              1348,
+              19997
+            ],
+            "documents": 4703,
+            "words": 1356746,
+            "characters": 11535346,
+            "new_tokens": 3232756,
+            "base_tokens": 4084484,
+            "new_tokens_per_word": 2.3827274965247733,
+            "base_tokens_per_word": 3.0105001230886255,
+            "token_reduction": 0.2085276867286051,
+            "token_reduction_95ci": [
+              0.1998185981685246,
+              0.21714855506411254
+            ],
+            "median_document_tokens_per_word": {
+              "new": 1.819905213270142,
+              "base": 3.3981481481481484
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.4448224537529237,
+              "equal_fraction": 0.5551775462470764,
+              "new_more_fraction": 0.0
+            }
+          }
+        }
+      }
+    },
+    "uz-crawl control": {
+      "domain_status": "training-source control",
+      "source": "tahrirchi/uz-crawl local training source",
+      "sampling": {
+        "records_scanned": 1618035,
+        "valid_records": 1441908,
+        "sampled_records": 100000,
+        "selected_duplicate_occurrences_skipped": 174,
+        "records_truncated_before_sampling": 9216
+      },
+      "result": {
+        "documents": 100000,
+        "words": 20825680,
+        "characters": 174743319,
+        "new_tokens": 48327821,
+        "base_tokens": 69191245,
+        "new_tokens_per_word": 2.3205878991706395,
+        "base_tokens_per_word": 3.3224002769657464,
+        "token_reduction": 0.3015327155914018,
+        "token_reduction_95ci": [
+          0.3001693372220501,
+          0.302962741125678
+        ],
+        "median_document_tokens_per_word": {
+          "new": 2.0866859344894024,
+          "base": 3.2613636363636362
+        },
+        "document_token_count_comparison": {
+          "new_fewer_fraction": 0.86481,
+          "equal_fraction": 0.13306,
+          "new_more_fraction": 0.00213
+        },
+        "document_length_chars": {
+          "min": 250,
+          "median": 1036,
+          "p90": 3408,
+          "p99": 14367,
+          "max": 19999
+        },
+        "by_script": {
+          "latin_dominant": {
+            "documents": 82818,
+            "words": 17899459,
+            "characters": 152194135,
+            "new_tokens": 37942391,
+            "base_tokens": 58788459,
+            "new_tokens_per_word": 2.119750714253431,
+            "base_tokens_per_word": 3.284370717573084,
+            "token_reduction": 0.3545945642154015,
+            "token_reduction_95ci": [
+              0.35375779400897717,
+              0.35539015324512163
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.0297619047619047,
+              "base": 3.2142857142857144
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 1.0,
+              "equal_fraction": 0.0,
+              "new_more_fraction": 0.0
+            }
+          },
+          "cyrillic_dominant": {
+            "documents": 17182,
+            "words": 2926221,
+            "characters": 22549184,
+            "new_tokens": 10385430,
+            "base_tokens": 10402786,
+            "new_tokens_per_word": 3.5490928402195188,
+            "base_tokens_per_word": 3.5550240395376838,
+            "token_reduction": 0.0016683992153639915,
+            "token_reduction_95ci": [
+              0.0013899888640842822,
+              0.001964975103579841
+            ],
+            "median_document_tokens_per_word": {
+              "new": 3.542372881355932,
+              "base": 3.5457593062605754
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.21318822023047376,
+              "equal_fraction": 0.7744150855546502,
+              "new_more_fraction": 0.012396694214876033
+            }
+          }
+        },
+        "by_document_length_quartile": {
+          "q1": {
+            "char_bounds": [
+              250,
+              642
+            ],
+            "documents": 25014,
+            "words": 1443765,
+            "characters": 11560305,
+            "new_tokens": 3580625,
+            "base_tokens": 4717032,
+            "new_tokens_per_word": 2.4800608132209883,
+            "base_tokens_per_word": 3.2671743670195634,
+            "token_reduction": 0.2409156859652426,
+            "token_reduction_95ci": [
+              0.2388878125954489,
+              0.243056097563355
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.2058823529411766,
+              "base": 3.2567567567567566
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.7540577276725035,
+              "equal_fraction": 0.24566242903973776,
+              "new_more_fraction": 0.00027984328775885504
+            }
+          },
+          "q2": {
+            "char_bounds": [
+              643,
+              1036
+            ],
+            "documents": 25032,
+            "words": 2534284,
+            "characters": 20698884,
+            "new_tokens": 5852869,
+            "base_tokens": 8233439,
+            "new_tokens_per_word": 2.3094763649220056,
+            "base_tokens_per_word": 3.248822547117845,
+            "token_reduction": 0.2891343459276251,
+            "token_reduction_95ci": [
+              0.2871033465654421,
+              0.2910416333852472
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.0697674418604652,
+              "base": 3.2268041237113403
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.8623761585170981,
+              "equal_fraction": 0.13690476190476192,
+              "new_more_fraction": 0.0007190795781399808
+            }
+          },
+          "q3": {
+            "char_bounds": [
+              1037,
+              1827
+            ],
+            "documents": 24959,
+            "words": 4090354,
+            "characters": 34187499,
+            "new_tokens": 9144176,
+            "base_tokens": 13420837,
+            "new_tokens_per_word": 2.235546361024009,
+            "base_tokens_per_word": 3.281094252477903,
+            "token_reduction": 0.31865829232558296,
+            "token_reduction_95ci": [
+              0.3168107024780933,
+              0.3205574932440705
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.051851851851852,
+              "base": 3.261904761904762
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.9184662847069194,
+              "equal_fraction": 0.08033174406025882,
+              "new_more_fraction": 0.0012019712328218278
+            }
+          },
+          "q4": {
+            "char_bounds": [
+              1828,
+              19999
+            ],
+            "documents": 24995,
+            "words": 12757277,
+            "characters": 108296631,
+            "new_tokens": 29750151,
+            "base_tokens": 42819937,
+            "new_tokens_per_word": 2.3320141908026297,
+            "base_tokens_per_word": 3.3565107193329737,
+            "token_reduction": 0.30522665178138864,
+            "token_reduction_95ci": [
+              0.3030310155219502,
+              0.30739078599543024
+            ],
+            "median_document_tokens_per_word": {
+              "new": 2.07344398340249,
+              "base": 3.2913907284768213
+            },
+            "document_token_count_comparison": {
+              "new_fewer_fraction": 0.924504900980196,
+              "equal_fraction": 0.0691738347669534,
+              "new_more_fraction": 0.00632126425285057
+            }
+          }
+        }
+      }
+    }
+  }
+}

fertility_large_20260825.md

@@ -0,0 +1,21 @@
+# NeuronAI-2B large tokenizer fertility evaluation
+
+Generated: 2026-08-24T19:25:09+00:00
+
+Fertility is the number of tokenizer tokens per whitespace-delimited word; lower is better.
+The confidence interval is a paired 5,000-replicate bootstrap over 1,000 deterministic document buckets.
+
+| Corpus | Domain status | Documents | Words | NeuronAI-2B | Qwen3.5-2B | Token reduction (95% CI) |
+| --- | --- | ---: | ---: | ---: | ---: | ---: |
+| OSCAR Uzbek | held-out web source | 100,000 | 7,618,770 | 2.0304 | 3.3639 | 39.64% (39.57%–39.71%) |
+| Uzbek legal corpus | held-out legal source/domain | 18,832 | 2,534,566 | 2.3747 | 2.9705 | 20.06% (19.55%–20.57%) |
+| uz-crawl control | training-source control | 100,000 | 20,825,680 | 2.3206 | 3.3224 | 30.15% (30.02%–30.30%) |
+
+## Interpretation
+
+- The 2B and 4B evaluated tokenizer pairs are byte-identical, so their token counts are exactly identical. SHA-256 fingerprints are recorded in the JSON result.
+- The held-out-source corpora are the fairness readout; the training-source corpus is retained only as a control.
+- Exact web overlap cannot be ruled out for OSCAR because both corpora ultimately derive from public web data.
+- OSCAR was checked in a post-hoc weak-token coverage analysis but contributed no tokenizer-training rows.
+- The legal corpus is absent from the tokenizer and CPT source manifests and is the cleanest source-and-domain holdout in this run.
+- Results use raw source text without apostrophe normalization or other model-specific preprocessing.