README.md

---
library_name: transformers
license: apache-2.0
datasets:
- TFLai/Turkish-Alpaca
language:
- tr
---

# Model Card: SykoLLM-V1-Turkish

SykoLLM-V1.2-Turkish-Instruct is a custom-architected, lightweight Large Language Model (LLM) designed specifically for Turkish conversational tasks. Unlike standard pre-built models, this version features a custom configuration optimized for speed and efficiency in low-resource environments.

## Model Description

* **Developed by:** syko818121
* **Model Name:** SykoLLM-V1.2-Turkish-Instruct
* **Model Type:** Causal Decoder-Only Custom Architecture
* **Language:** Turkish
* **Parameters:** ~50.8 Million
* **Training Data:** Turkish Wikipedia + Custom High-Quality Chat Dataset

## Architectural Specs

This model uses a custom configuration designed for Turkish linguistics:

* **Vocabulary Size:** 50,257
* **Hidden Dimension (n_embd):** 512
* **Number of Layers:** 8
* **Attention Heads:** 8
* **Context Window:** 512 tokens

## Fine-Tuning & Conversation Style

The model was fine-tuned on a high-quality, curated Turkish dataset  to ensure natural, human-like responses. The training data distribution was carefully balanced:

* 
**Greetings & Daily Talk (40%):** Natural openings and casual conversation.


* 
**Direct Question-Answering (30%):** Short and concise answers to general knowledge queries.


* 
**Brief Explanations (20%):** Simplified definitions for complex concepts.


* 
**Slang & Short Inputs (10%):** Robustness against one-word or incomplete messages.



## Usage

You can load and test SykoLLM-V1-Turkish using the following snippet:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "syko818121/SykoLLM-V1-Turkish"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

prompt = "<user> Selam, naber?<assistant>"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50, pad_token_id=tokenizer.eos_token_id)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

```

## Training Configuration

* **Learning Rate:** 5e-5
* 
**Scheduler:** Cosine 


* **Epochs:** 15
* **Batch Size:** 4
* **Precision:** FP16 (Mixed Precision)

## Limitations

* **Size:** As a 50.8M parameter model, it is a "micro-LLM." It excels at short chats but may hallucinate on highly complex logical tasks.
* **Response Length:** The model is intentionally biased toward concise and direct answers rather than long-form essays.

---