| --- |
| license: apache-2.0 |
| language: |
| - tr |
| pipeline_tag: text-generation |
| tags: |
| - slm |
| - base-model |
| - causal-lm |
| - pre-trained |
| - tr-llm |
| - Ahıska |
| - AhiskaTurks |
| - MeskhetianTurks |
| - AhıskaTürkleri |
| library_name: transformers |
|
|
| --- |
| |
| # AhiskaAI-65m-IT-v0.2 |
|
|
| AhiskaAI-65m-IT-v0.2 is the instruction-tuned version of our 65M parameter Small Language Model. Fine-tuned on a curated Turkish instruction dataset, it is designed to function as a lightweight conversational AI assistant while maintaining fast inference on resource-constrained hardware. |
|
|
| **Base Model:** AhiskaAI-65m-Base-v0.2 |
|
|
| --- |
|
|
| ## Model Details |
|
|
| - **Architecture:** Llama-based architecture. |
| - **Fine-tuning:** Supervised Fine-Tuning (SFT). |
| - **Format:** ChatML. |
| - **Parameters:** 65M. |
| - **Context Window:** 1024 tokens. |
| - **Tokenizer:** Custom BPE Tokenizer (Vocabulary Size: 32,000). |
| - **Training Framework:** PyTorch & Transformers. |
| - **Hardware:** NVIDIA RTX 4050 6GB Laptop GPU. |
|
|
| --- |
|
|
| ## Fine-tuning Dataset |
|
|
| The model was fine-tuned using a curated Turkish instruction dataset designed to improve conversational ability and instruction following. |
|
|
| The dataset focuses on: |
|
|
| - Question answering |
| - General conversation |
| - Summarization |
| - Text generation |
| - Turkish instruction following |
|
|
| --- |
|
|
| ## Design Goal |
|
|
| The 65M-IT model is designed as the lightweight conversational member of the AhiskaAI v0.2 family. |
|
|
| Its primary goals are: |
|
|
| - Basic Turkish instruction following. |
| - Fast conversational inference. |
| - Low-resource deployment. |
| - A compact research baseline for future alignment methods. |
|
|
| --- |
|
|
| ## Training Logs |
|
|
|  |
|
|
| *The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-65m-IT-v0.2.* |
|
|
| --- |
|
|
| ## Usage (ChatML) |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-65m-IT-v0.2") |
| tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-65m-IT-v0.2") |
| |
| SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın." |
| |
| prompt = ( |
| f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n" |
| f"<|im_start|>user\nMerhaba<|im_end|>\n" |
| f"<|im_start|>assistant\n" |
| ) |
| |
| inputs = tokenizer(prompt, return_tensors="pt") |
| |
| outputs = model.generate(**inputs, max_new_tokens=200) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| --- |
|
|
| ## Known Limitations |
|
|
| - Limited factual knowledge due to model size. |
| - Optimized primarily for Turkish. |
| - Context window limited to 1024 tokens. |
| - May generate inaccurate or incomplete responses on complex topics. |
|
|
| --- |
|
|
| ## Future Plans |
|
|
| - DPO preference alignment. |
| - Improved instruction datasets. |
| - Future AhiskaAI v0.3 releases. |
|
|
| --- |
|
|
| ## About AhiskaAI |
|
|
| AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch. |
|
|
| Follow us on Hugging Face for updates and future releases. |
