|
|
--- |
|
|
library_name: transformers |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- TFLai/Turkish-Alpaca |
|
|
language: |
|
|
- tr |
|
|
--- |
|
|
|
|
|
# Model Card: SykoLLM-V1-Turkish |
|
|
|
|
|
SykoLLM-V1.2-Turkish-Instruct is a custom-architected, lightweight Large Language Model (LLM) designed specifically for Turkish conversational tasks. Unlike standard pre-built models, this version features a custom configuration optimized for speed and efficiency in low-resource environments. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
* **Developed by:** syko818121 |
|
|
* **Model Name:** SykoLLM-V1.2-Turkish-Instruct |
|
|
* **Model Type:** Causal Decoder-Only Custom Architecture |
|
|
* **Language:** Turkish |
|
|
* **Parameters:** ~50.8 Million |
|
|
* **Training Data:** Turkish Wikipedia + Custom High-Quality Chat Dataset |
|
|
|
|
|
## Architectural Specs |
|
|
|
|
|
This model uses a custom configuration designed for Turkish linguistics: |
|
|
|
|
|
* **Vocabulary Size:** 50,257 |
|
|
* **Hidden Dimension (n_embd):** 512 |
|
|
* **Number of Layers:** 8 |
|
|
* **Attention Heads:** 8 |
|
|
* **Context Window:** 512 tokens |
|
|
|
|
|
## Fine-Tuning & Conversation Style |
|
|
|
|
|
The model was fine-tuned on a high-quality, curated Turkish dataset to ensure natural, human-like responses. The training data distribution was carefully balanced: |
|
|
|
|
|
* |
|
|
**Greetings & Daily Talk (40%):** Natural openings and casual conversation. |
|
|
|
|
|
|
|
|
* |
|
|
**Direct Question-Answering (30%):** Short and concise answers to general knowledge queries. |
|
|
|
|
|
|
|
|
* |
|
|
**Brief Explanations (20%):** Simplified definitions for complex concepts. |
|
|
|
|
|
|
|
|
* |
|
|
**Slang & Short Inputs (10%):** Robustness against one-word or incomplete messages. |
|
|
|
|
|
|
|
|
|
|
|
## Usage |
|
|
|
|
|
You can load and test SykoLLM-V1-Turkish using the following snippet: |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model_id = "syko818121/SykoLLM-V1-Turkish" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True) |
|
|
|
|
|
prompt = "<user> Selam, naber?<assistant>" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=50, pad_token_id=tokenizer.eos_token_id) |
|
|
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
|
|
|
``` |
|
|
|
|
|
## Training Configuration |
|
|
|
|
|
* **Learning Rate:** 5e-5 |
|
|
* |
|
|
**Scheduler:** Cosine |
|
|
|
|
|
|
|
|
* **Epochs:** 15 |
|
|
* **Batch Size:** 4 |
|
|
* **Precision:** FP16 (Mixed Precision) |
|
|
|
|
|
## Limitations |
|
|
|
|
|
* **Size:** As a 50.8M parameter model, it is a "micro-LLM." It excels at short chats but may hallucinate on highly complex logical tasks. |
|
|
* **Response Length:** The model is intentionally biased toward concise and direct answers rather than long-form essays. |
|
|
|
|
|
--- |
