| --- |
| license: apache-2.0 |
| datasets: |
| - HuggingFaceFW/fineweb |
| - yashmarathe/OpenMathReasoning |
| language: |
| - en |
| pipeline_tag: text-classification |
| tags: |
| - llm |
| - azerbaijan |
| - azerbaijani-llm-model |
| --- |
| <p align="center"> |
| <img src="./assets/khazri-2-preview-banner.png" alt="Khazri 2 Preview — flagship open-weight language model" width="100%"> |
| </p> |
|
|
| # Khazri 2 Preview |
|
|
| **Khazri 2 Preview** is the flagship open-weight release in the current Khazri family. It combines the Khazri model with the project's **SYNAPSE** architecture and route-aware inference system for research, experimentation and controlled integration. |
|
|
| [Hugging Face](https://huggingface.co/Yusiko/khazri-2) · [Khazri](https://khazri.dev) · [Contact](mailto:contact@khazri.dev) |
|
|
| > **Preview release:** Validate this model on your own tasks before production use. It should not be the sole basis for high-impact decisions. |
|
|
| ## At a glance |
|
|
| | Item | Detail | |
| | --- | --- | |
| | Model | Khazri 2 Preview | |
| | Family position | Flagship model in the Khazri 2 generation | |
| | Parameters | ~250M; the release config is the final source of truth | |
| | Status | Open weights on Hugging Face | |
| | Architecture | Khazri model with SYNAPSE components | |
| | Safe export context limit | 512 tokens | |
| | Default maximum generation | 128 new tokens | |
| | Loading | Transformers with custom model code | |
| | Web-search route | Disabled by default | |
|
|
| ## SYNAPSE-aware inference |
|
|
| The release includes the configuration, tokenizer, safetensor weights and the custom SYNAPSE model files required for the inference stack: |
|
|
| ~~~text |
| config.json |
| generation_config.json |
| model.safetensors |
| tokenizer.json |
| tokenizer_config.json |
| modeling_synapse.py |
| synapse_controller.py |
| inference.py |
| ~~~ |
|
|
| The route system supports: |
|
|
| - NSR_CALCULATOR for safe numeric calculation; |
| - MATH_SOLVER for symbolic mathematics; |
| - TMS_CONTEXT for context-grounded responses; |
| - UQM_ABSTAIN when information is insufficient or inappropriate; |
| - WEB_SEARCH, disabled by default; and |
| - MODEL_FALLBACK for standard model generation. |
|
|
| Routing is implementation-dependent. Inspect the custom Python files before enabling remote code and pin a specific model revision in production. |
|
|
| ## Training-data documentation |
|
|
| Khazri 2 Preview does not state a source-level corpus mix in this model card. Do not assume that it uses the same sources, proportions or post-training process as Khazri 2 Mini. |
|
|
| For a particular release, use the data documentation and model revision published with the [Khazri 2 Hugging Face repository](https://huggingface.co/Yusiko/khazri-2). A complete data card should identify source datasets and licences, language coverage, filtering and deduplication steps, pre-training and instruction-tuning material, known limitations and revision-specific evaluation. |
|
|
| ## Installation |
|
|
| ~~~bash |
| pip install -U torch transformers accelerate safetensors |
| ~~~ |
|
|
| ## Quick start |
|
|
| ~~~python |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| MODEL_ID = "Yusiko/khazri-2" |
| |
| # Inspect the model repository before enabling custom remote code. |
| tokenizer = AutoTokenizer.from_pretrained( |
| MODEL_ID, |
| trust_remote_code=True, |
| use_fast=True, |
| ) |
| model = AutoModelForCausalLM.from_pretrained( |
| MODEL_ID, |
| torch_dtype="auto", |
| device_map="auto", |
| trust_remote_code=True, |
| ) |
| |
| prompt = "Summarize the following text in two concise sentences: ..." |
| inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=384) |
| inputs = {key: value.to(model.device) for key, value in inputs.items()} |
| |
| with torch.inference_mode(): |
| output = model.generate( |
| **inputs, |
| max_new_tokens=128, |
| do_sample=False, |
| pad_token_id=tokenizer.eos_token_id, |
| ) |
| |
| print(tokenizer.decode(output[0], skip_special_tokens=True)) |
| ~~~ |
|
|
| For this preview export, reserve generation tokens within the 512-token total context limit. For example, use up to 384 input tokens with 128 new tokens. |
|
|
| ## Compact-model comparison |
|
|
| The following values are project-provided internal results for Khazri 2 Preview across selected compact-model tests. Higher is better. |
|
|
| | Model | Parameters | Context extraction | Mixed speed/proxy | Arithmetic | Word problems | Abstention | |
| | --- | ---: | ---: | ---: | ---: | ---: | ---: | |
| | **Khazri 2 Preview** | ~250M | **100%** | **62%** | **99%** | **99%** | **97.4%** | |
| | Gemma 3 | 270M | 100% | 36% | 0% | 0% | 18% | |
| | Qwen 2.5 | 0.5B | 89% | 45% | 14% | 28% | 46% | |
| | Pythia | 160M | 22% | 10% | 0% | 2% | 1% | |
|
|
| These figures are not independently audited. They describe the reported test configuration only and do not establish general superiority or predict performance on every downstream task. Publish prompts, task definitions, scoring rules, model versions, seeds, hardware and full evaluation assets with future public benchmark claims. |
|
|
| ## Responsible use |
|
|
| Khazri 2 Preview may hallucinate, make reasoning errors, reflect training-data bias or abstain incorrectly. Use retrieval, verification and human review for consequential work. Do not rely on outputs alone for medical, legal, financial, security, safety, education, employment or other high-impact decisions. |
|
|
| ## Release notes |
|
|
| Use the files, revision and licence stated on the [Hugging Face model page](https://huggingface.co/Yusiko/khazri-2) as the source of truth for a specific release. |
|
|
| ## Roadmap |
|
|
| Khazri 2 Preview represents the current flagship stage. The next goal is **Khazri 3**: a model with a larger parameter count and stronger results. |
|
|
| ## Contact |
|
|
| To discuss research, integration or partnerships, visit [khazri.dev](https://khazri.dev) or email [contact@khazri.dev](mailto:contact@khazri.dev). |
