| | --- |
| | library_name: transformers |
| | license: apache-2.0 |
| | datasets: |
| | - TFLai/Turkish-Alpaca |
| | language: |
| | - tr |
| | --- |
| | |
| | # Model Card: SykoLLM-V1-Turkish |
| |
|
| | SykoLLM-V1.2-Turkish-Instruct is a custom-architected, lightweight Large Language Model (LLM) designed specifically for Turkish conversational tasks. Unlike standard pre-built models, this version features a custom configuration optimized for speed and efficiency in low-resource environments. |
| |
|
| | ## Model Description |
| |
|
| | * **Developed by:** syko818121 |
| | * **Model Name:** SykoLLM-V1.2-Turkish-Instruct |
| | * **Model Type:** Causal Decoder-Only Custom Architecture |
| | * **Language:** Turkish |
| | * **Parameters:** ~50.8 Million |
| | * **Training Data:** Turkish Wikipedia + Custom High-Quality Chat Dataset |
| |
|
| | ## Architectural Specs |
| |
|
| | This model uses a custom configuration designed for Turkish linguistics: |
| |
|
| | * **Vocabulary Size:** 50,257 |
| | * **Hidden Dimension (n_embd):** 512 |
| | * **Number of Layers:** 8 |
| | * **Attention Heads:** 8 |
| | * **Context Window:** 512 tokens |
| | |
| | ## Fine-Tuning & Conversation Style |
| | |
| | The model was fine-tuned on a high-quality, curated Turkish dataset to ensure natural, human-like responses. The training data distribution was carefully balanced: |
| | |
| | * |
| | **Greetings & Daily Talk (40%):** Natural openings and casual conversation. |
| | |
| | |
| | * |
| | **Direct Question-Answering (30%):** Short and concise answers to general knowledge queries. |
| | |
| | |
| | * |
| | **Brief Explanations (20%):** Simplified definitions for complex concepts. |
| | |
| | |
| | * |
| | **Slang & Short Inputs (10%):** Robustness against one-word or incomplete messages. |
| | |
| | |
| | |
| | ## Usage |
| | |
| | You can load and test SykoLLM-V1-Turkish using the following snippet: |
| | |
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model_id = "syko818121/SykoLLM-V1-Turkish" |
| | tokenizer = AutoTokenizer.from_pretrained(model_id) |
| | model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True) |
| | |
| | prompt = "<user> Selam, naber?<assistant>" |
| | inputs = tokenizer(prompt, return_tensors="pt") |
| | outputs = model.generate(**inputs, max_new_tokens=50, pad_token_id=tokenizer.eos_token_id) |
| | |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | |
| | ``` |
| | |
| | ## Training Configuration |
| | |
| | * **Learning Rate:** 5e-5 |
| | * |
| | **Scheduler:** Cosine |
| | |
| | |
| | * **Epochs:** 15 |
| | * **Batch Size:** 4 |
| | * **Precision:** FP16 (Mixed Precision) |
| | |
| | ## Limitations |
| | |
| | * **Size:** As a 50.8M parameter model, it is a "micro-LLM." It excels at short chats but may hallucinate on highly complex logical tasks. |
| | * **Response Length:** The model is intentionally biased toward concise and direct answers rather than long-form essays. |
| | |
| | --- |
