| --- |
| license: apache-2.0 |
| language: |
| - tr |
| pipeline_tag: text-generation |
| tags: |
| - instruct |
| - chat |
| - conversational |
| - fine-tuned |
| - slm |
| - llama |
| - tr-llm |
| - Ahiska |
| library_name: transformers |
| --- |
| |
|
|
| # AhiskaAI-308m-IT-v0.2 |
|
|
| AhiskaAI-308m-IT-v0.2 is the instruction-tuned version of our 308M parameter Small Language Model. Fine-tuned on 16,000+ curated Turkish instruction-response pairs, it is designed to provide stronger conversational ability and improved instruction following while remaining efficient enough to run on consumer hardware. |
|
|
| **Base Model:** AhiskaAI-308m-Base-v0.2 |
|
|
| --- |
|
|
| ## Model Details |
|
|
| - **Architecture:** Llama-based architecture. |
| - **Fine-tuning:** Supervised Fine-Tuning (SFT). |
| - **Format:** ChatML. |
| - **Parameters:** 308M. |
| - **Context Window:** 1024 tokens. |
| - **Tokenizer:** Custom BPE Tokenizer (Vocabulary Size: 32,000). |
| - **Training Framework:** PyTorch & Transformers. |
| - **Hardware:** NVIDIA RTX 4050 6GB Laptop GPU. |
|
|
| --- |
|
|
| ## Fine-tuning Dataset |
|
|
| The model was fine-tuned using more than **16,000** carefully curated Turkish instruction-response pairs. |
|
|
| The dataset includes tasks such as: |
|
|
| - Question answering |
| - General conversation |
| - Summarization |
| - Text generation |
| - Instruction following |
| - Basic reasoning |
|
|
| --- |
|
|
| ## Design Goal |
|
|
| The 308M-IT model serves as the flagship conversational model of the AhiskaAI v0.2 family. |
|
|
| Its primary objectives are: |
|
|
| - Improved Turkish instruction following. |
| - Better contextual understanding. |
| - More natural conversational responses. |
| - A strong research foundation for future preference alignment methods such as DPO. |
|
|
| --- |
|
|
| ## Training Logs |
|
|
|  |
|
|
| *The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-308m-IT-v0.2.* |
|
|
| --- |
|
|
| ## Usage (ChatML Format) |
|
|
| ### Recommended System Prompt |
|
|
| ``` |
| Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın. |
| ``` |
|
|
| ### Example Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2") |
| tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2") |
| |
| SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın." |
| |
| user_query = "Ahıska Türkleri hakkında bilgi verir misin?" |
| |
| prompt = ( |
| f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n" |
| f"<|im_start|>user\n{user_query}<|im_end|>\n" |
| f"<|im_start|>assistant\n" |
| ) |
| |
| inputs = tokenizer(prompt, return_tensors="pt") |
| |
| outputs = model.generate(**inputs, max_new_tokens=200) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| --- |
|
|
| ## Known Limitations |
|
|
| - Optimized primarily for Turkish. |
| - Context window is limited to 1024 tokens. |
| - Factual accuracy is still limited by model size and pretraining data. |
| - May generate incorrect or incomplete responses on complex reasoning tasks. |
|
|
| --- |
|
|
| ## Future Plans |
|
|
| - Preference alignment using DPO. |
| - Larger, higher-quality Turkish datasets. |
| - Expanded evaluation benchmarks. |
| - Future AhiskaAI v0.3 model family. |
|
|
| --- |
|
|
| ## About AhiskaAI |
|
|
| AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch. |
|
|
| Follow us on Hugging Face for updates and future releases. |
