Update README.md
Browse files
README.md
CHANGED
|
@@ -1,35 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
<p align="center">
|
| 2 |
<img src="./assets/khazri-2-preview-banner.png" alt="Khazri 2 Preview — flagship open-weight language model" width="100%">
|
| 3 |
</p>
|
| 4 |
|
| 5 |
-
<p align="center">
|
| 6 |
-
<img src="./assets/khazri-wordmark.png" alt="Khazri 2" width="280">
|
| 7 |
-
</p>
|
| 8 |
-
|
| 9 |
# Khazri 2 Preview
|
| 10 |
|
| 11 |
-
**Khazri 2 Preview** is the flagship
|
| 12 |
|
| 13 |
-
[
|
| 14 |
|
| 15 |
-
> **Preview release:**
|
| 16 |
|
| 17 |
## At a glance
|
| 18 |
|
| 19 |
| Item | Detail |
|
| 20 |
| --- | --- |
|
| 21 |
| Model | Khazri 2 Preview |
|
| 22 |
-
| Family position | Flagship model in the
|
| 23 |
-
| Parameters | ~250M
|
|
|
|
| 24 |
| Architecture | Khazri model with SYNAPSE components |
|
| 25 |
-
|
|
| 26 |
| Default maximum generation | 128 new tokens |
|
| 27 |
| Loading | Transformers with custom model code |
|
| 28 |
-
| Web-search route | Disabled by default
|
| 29 |
|
| 30 |
## SYNAPSE-aware inference
|
| 31 |
|
| 32 |
-
The
|
| 33 |
|
| 34 |
~~~text
|
| 35 |
config.json
|
|
@@ -42,33 +52,22 @@ synapse_controller.py
|
|
| 42 |
inference.py
|
| 43 |
~~~
|
| 44 |
|
| 45 |
-
The
|
| 46 |
-
|
| 47 |
-
- <code>NSR_CALCULATOR</code> for safe numeric calculation;
|
| 48 |
-
- <code>MATH_SOLVER</code> for symbolic math tasks;
|
| 49 |
-
- <code>TMS_CONTEXT</code> for context-grounded answers;
|
| 50 |
-
- <code>UQM_ABSTAIN</code> for abstention when information is insufficient or inappropriate;
|
| 51 |
-
- <code>WEB_SEARCH</code>, disabled by default; and
|
| 52 |
-
- <code>MODEL_FALLBACK</code> for standard generation.
|
| 53 |
-
|
| 54 |
-
The exact routing decision is implementation-dependent. Inspect the shipped Python files before enabling remote code, and pin a revision in production.
|
| 55 |
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
-
|
| 59 |
|
| 60 |
-
|
| 61 |
|
| 62 |
-
|
| 63 |
|
| 64 |
-
|
| 65 |
-
2. language coverage and source proportions;
|
| 66 |
-
3. collection dates, filtering and deduplication methods;
|
| 67 |
-
4. known limitations, bias and personal-data controls;
|
| 68 |
-
5. pre-training versus instruction-tuning sources; and
|
| 69 |
-
6. the exact relationship, if any, between the Preview checkpoint and the Mini training corpus.
|
| 70 |
-
|
| 71 |
-
This distinction protects both users and the project: it makes the open-weight release auditable without overstating what the supplied files prove.
|
| 72 |
|
| 73 |
## Installation
|
| 74 |
|
|
@@ -78,15 +77,13 @@ pip install -U torch transformers accelerate safetensors
|
|
| 78 |
|
| 79 |
## Quick start
|
| 80 |
|
| 81 |
-
Replace <code>YOUR_ORG/Khazri-2-Preview</code> with the final Hugging Face repository ID. The release must include the custom SYNAPSE Python files before the following example can work.
|
| 82 |
-
|
| 83 |
~~~python
|
| 84 |
import torch
|
| 85 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 86 |
|
| 87 |
-
MODEL_ID = "
|
| 88 |
|
| 89 |
-
#
|
| 90 |
tokenizer = AutoTokenizer.from_pretrained(
|
| 91 |
MODEL_ID,
|
| 92 |
trust_remote_code=True,
|
|
@@ -114,11 +111,11 @@ with torch.inference_mode():
|
|
| 114 |
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
| 115 |
~~~
|
| 116 |
|
| 117 |
-
|
| 118 |
|
| 119 |
## Compact-model comparison
|
| 120 |
|
| 121 |
-
The following
|
| 122 |
|
| 123 |
| Model | Parameters | Context extraction | Mixed speed/proxy | Arithmetic | Word problems | Abstention |
|
| 124 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
@@ -127,30 +124,20 @@ The following numbers are the project-provided **Khazri 2 Preview** comparison a
|
|
| 127 |
| Qwen 2.5 | 0.5B | 89% | 45% | 14% | 28% | 46% |
|
| 128 |
| Pythia | 160M | 22% | 10% | 0% | 2% | 1% |
|
| 129 |
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
These are internal preview results, not an independently audited benchmark. The supplied material does not include all prompts, task definitions, scoring rules, versions, seeds, hardware, sampling settings or full evaluation set. They therefore describe the reported tests only; they do not prove general superiority or predict performance on every downstream task. Reproducible evaluation assets should accompany any future public benchmark announcement.
|
| 133 |
|
| 134 |
## Responsible use
|
| 135 |
|
| 136 |
-
Khazri 2 Preview may hallucinate, make reasoning errors, reflect training-data bias or abstain incorrectly. Use retrieval, verification and human review for consequential work. Do not rely on outputs alone for medical, legal, financial, security, safety, education, employment or other high-impact decisions.
|
| 137 |
-
|
| 138 |
-
## Release checklist
|
| 139 |
|
| 140 |
-
|
| 141 |
|
| 142 |
-
|
| 143 |
-
- [ ] ship the weights, config, tokenizer and all required SYNAPSE files;
|
| 144 |
-
- [ ] add a <code>LICENSE</code> file; no licence is claimed in this README before one is chosen;
|
| 145 |
-
- [ ] publish <code>DATASET.md</code> with source/licence information;
|
| 146 |
-
- [ ] publish an evaluation card with prompts, versions and scoring code;
|
| 147 |
-
- [ ] pin a version/revision and publish file checksums;
|
| 148 |
-
- [ ] test fresh installation in a clean environment.
|
| 149 |
|
| 150 |
## Roadmap
|
| 151 |
|
| 152 |
-
Khazri 2 Preview represents the current flagship stage. The next goal is **Khazri 3**: a model with a larger parameter count and stronger results
|
| 153 |
|
| 154 |
## Contact
|
| 155 |
|
| 156 |
-
To discuss research,
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
datasets:
|
| 4 |
+
- HuggingFaceFW/fineweb
|
| 5 |
+
- yashmarathe/OpenMathReasoning
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
+
pipeline_tag: text-classification
|
| 9 |
+
tags:
|
| 10 |
+
- llm
|
| 11 |
+
- azerbaijan
|
| 12 |
+
- azerbaijani-llm-model
|
| 13 |
+
---
|
| 14 |
<p align="center">
|
| 15 |
<img src="./assets/khazri-2-preview-banner.png" alt="Khazri 2 Preview — flagship open-weight language model" width="100%">
|
| 16 |
</p>
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
# Khazri 2 Preview
|
| 19 |
|
| 20 |
+
**Khazri 2 Preview** is the flagship open-weight release in the current Khazri family. It combines the Khazri model with the project's **SYNAPSE** architecture and route-aware inference system for research, experimentation and controlled integration.
|
| 21 |
|
| 22 |
+
[Hugging Face](https://huggingface.co/Yusiko/khazri-2) · [Khazri](https://khazri.dev) · [Contact](mailto:contact@khazri.dev)
|
| 23 |
|
| 24 |
+
> **Preview release:** Validate this model on your own tasks before production use. It should not be the sole basis for high-impact decisions.
|
| 25 |
|
| 26 |
## At a glance
|
| 27 |
|
| 28 |
| Item | Detail |
|
| 29 |
| --- | --- |
|
| 30 |
| Model | Khazri 2 Preview |
|
| 31 |
+
| Family position | Flagship model in the Khazri 2 generation |
|
| 32 |
+
| Parameters | ~250M; the release config is the final source of truth |
|
| 33 |
+
| Status | Open weights on Hugging Face |
|
| 34 |
| Architecture | Khazri model with SYNAPSE components |
|
| 35 |
+
| Safe export context limit | 512 tokens |
|
| 36 |
| Default maximum generation | 128 new tokens |
|
| 37 |
| Loading | Transformers with custom model code |
|
| 38 |
+
| Web-search route | Disabled by default |
|
| 39 |
|
| 40 |
## SYNAPSE-aware inference
|
| 41 |
|
| 42 |
+
The release includes the configuration, tokenizer, safetensor weights and the custom SYNAPSE model files required for the inference stack:
|
| 43 |
|
| 44 |
~~~text
|
| 45 |
config.json
|
|
|
|
| 52 |
inference.py
|
| 53 |
~~~
|
| 54 |
|
| 55 |
+
The route system supports:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
+
- NSR_CALCULATOR for safe numeric calculation;
|
| 58 |
+
- MATH_SOLVER for symbolic mathematics;
|
| 59 |
+
- TMS_CONTEXT for context-grounded responses;
|
| 60 |
+
- UQM_ABSTAIN when information is insufficient or inappropriate;
|
| 61 |
+
- WEB_SEARCH, disabled by default; and
|
| 62 |
+
- MODEL_FALLBACK for standard model generation.
|
| 63 |
|
| 64 |
+
Routing is implementation-dependent. Inspect the custom Python files before enabling remote code and pin a specific model revision in production.
|
| 65 |
|
| 66 |
+
## Training-data documentation
|
| 67 |
|
| 68 |
+
Khazri 2 Preview does not state a source-level corpus mix in this model card. Do not assume that it uses the same sources, proportions or post-training process as Khazri 2 Mini.
|
| 69 |
|
| 70 |
+
For a particular release, use the data documentation and model revision published with the [Khazri 2 Hugging Face repository](https://huggingface.co/Yusiko/khazri-2). A complete data card should identify source datasets and licences, language coverage, filtering and deduplication steps, pre-training and instruction-tuning material, known limitations and revision-specific evaluation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
## Installation
|
| 73 |
|
|
|
|
| 77 |
|
| 78 |
## Quick start
|
| 79 |
|
|
|
|
|
|
|
| 80 |
~~~python
|
| 81 |
import torch
|
| 82 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 83 |
|
| 84 |
+
MODEL_ID = "Yusiko/khazri-2"
|
| 85 |
|
| 86 |
+
# Inspect the model repository before enabling custom remote code.
|
| 87 |
tokenizer = AutoTokenizer.from_pretrained(
|
| 88 |
MODEL_ID,
|
| 89 |
trust_remote_code=True,
|
|
|
|
| 111 |
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
| 112 |
~~~
|
| 113 |
|
| 114 |
+
For this preview export, reserve generation tokens within the 512-token total context limit. For example, use up to 384 input tokens with 128 new tokens.
|
| 115 |
|
| 116 |
## Compact-model comparison
|
| 117 |
|
| 118 |
+
The following values are project-provided internal results for Khazri 2 Preview across selected compact-model tests. Higher is better.
|
| 119 |
|
| 120 |
| Model | Parameters | Context extraction | Mixed speed/proxy | Arithmetic | Word problems | Abstention |
|
| 121 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
|
|
| 124 |
| Qwen 2.5 | 0.5B | 89% | 45% | 14% | 28% | 46% |
|
| 125 |
| Pythia | 160M | 22% | 10% | 0% | 2% | 1% |
|
| 126 |
|
| 127 |
+
These figures are not independently audited. They describe the reported test configuration only and do not establish general superiority or predict performance on every downstream task. Publish prompts, task definitions, scoring rules, model versions, seeds, hardware and full evaluation assets with future public benchmark claims.
|
|
|
|
|
|
|
| 128 |
|
| 129 |
## Responsible use
|
| 130 |
|
| 131 |
+
Khazri 2 Preview may hallucinate, make reasoning errors, reflect training-data bias or abstain incorrectly. Use retrieval, verification and human review for consequential work. Do not rely on outputs alone for medical, legal, financial, security, safety, education, employment or other high-impact decisions.
|
|
|
|
|
|
|
| 132 |
|
| 133 |
+
## Release notes
|
| 134 |
|
| 135 |
+
Use the files, revision and licence stated on the [Hugging Face model page](https://huggingface.co/Yusiko/khazri-2) as the source of truth for a specific release.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
|
| 137 |
## Roadmap
|
| 138 |
|
| 139 |
+
Khazri 2 Preview represents the current flagship stage. The next goal is **Khazri 3**: a model with a larger parameter count and stronger results.
|
| 140 |
|
| 141 |
## Contact
|
| 142 |
|
| 143 |
+
To discuss research, integration or partnerships, visit [khazri.dev](https://khazri.dev) or email [contact@khazri.dev](mailto:contact@khazri.dev).
|