library_name: transformers
tags:
- chemistry
- biology
- finance
- legal
- music
- art
- code
- climate
- medical
- agent
- text-generation-inference
- Duchifat-2
- conversational
- chat
- SFT
license: apache-2.0
language:
- he
- en
base_model:
- Raziel1234/Duchifat-2
pipeline_tag: text-generation
metrics:
- brier_score
🕊️ Duchifat-2.2-Instruct
Duchifat-2.2-Instruct is a fine-tuned version of the original Duchifat-2 base model. While this specific version is an optimized Instruct/Chat model, the underlying base architecture and weights were developed and trained from scratch by Raziel.
🚀 Lineage & Development
- Base Model (Duchifat-2): Built and pre-trained from scratch on 3.27 Billion tokens (50/50 Hebrew-English C4 dataset). It features 136M parameters and was designed to establish a native Hebrew reasoning foundation.
- Version 2.2 (Instruct): A refined fine-tuned version (SFT) designed to transform the base capabilities into a quirky, safe, and highly responsive conversational agent.
Key Features:
- Native Hebrew Foundation: Unlike models that adapt English weights, Duchifat was born in Hebrew using the DictaLM tokenizer, ensuring high efficiency and natural linguistic flow.
- Compact Power: At only 136M parameters, it delivers impressive performance while remaining small enough for edge deployment and low-latency applications.
- Quirky & Human-like: The SFT process focused on giving the model a distinct personality—witty and engaging rather than robotic.
- Safety Integrated: Built-in guardrails ensure the model remains professional and refuses to engage with profanity or offensive prompts.
📊 Benchmark Results (Zero-Shot)
Tested using manual prompt formatting to accurately reflect real-world chat performance.
| Task | Version | Filter | n-shot | Metric | Value | Stderr |
|---|---|---|---|---|---|---|
| piqa | 1 | none | 0 | acc | 0.70 | ± 0.1528 |
| piqa | 1 | none | 0 | acc_norm | 0.70 | ± 0.1528 |
| hellaswag | 1 | none | 0 | acc | 0.40 | ± 0.1633 |
| hellaswag | 1 | none | 0 | acc_norm | 0.40 | ± 0.1633 |
| winogrande | 1 | none | 0 | acc | 0.40 | ± 0.1633 |
| arc_easy | 1 | none | 0 | acc | 0.10 | ± 0.1000 |
| arc_easy | 1 | none | 0 | acc_norm | 0.10 | ± 0.1000 |
🛠️ Technical Specifications
- Parameters: 136M
- Base Pre-training Data: 3.27B tokens (C4 Hebrew/English)
- Tokenizer: DictaLM (Hebrew optimized)
- Context Window: 1024 tokens
💡 How to Use
Use the following instruction format to trigger the Instruct-tuned behavior:
Prompt Template:
<|instruction|>
{user_query}
<|assistant|>
Example Usage:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "razielAI/Duchifat-2.2-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to("cuda")
prompt = "<|instruction|>\nשלום!\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))
⚠️ Limitations
Duchifat-2.2 is a lightweight model. It excels at conversational tasks, social media content, and short-form text generation. It is not designed for complex mathematical proofs or extensive coding sessions.
🕊️ About the Duchifat Project
The Duchifat (Hoopoe) project is dedicated to creating efficient, open-source AI with a native understanding of the Hebrew language and culture.