Update README.md

README.md

@@ -1,3 +1,59 @@
----
-license: apache-2.0
----
+---
+language:
+- tr
+license: apache-2.0
+library_name: transformers
+tags:
+- llama-3
+- turkish
+- tiny-llama
+- scratch-build
+datasets:
+- TFLai/Turkish-Alpaca
+metrics:
+- loss
+model_type: llama
+---
+
+# Llama-TR-Mini (9M Parameters)
+
+Llama-TR-Mini is an experimental, ultra-lightweight Turkish language model with **9.3 million parameters**, trained from scratch using the Llama 3 architecture. 
+
+This project was developed to explore the limits of small-scale language modeling and to understand the end-to-end pre-training/fine-tuning pipeline on consumer-grade hardware (Apple Silicon).
+
+## Model Specifications
+- **Architecture:** Llama 3
+- **Parameters:** 9,343,232
+- **Hidden Size:** 256
+- **Intermediate Size:** 512
+- **Number of Layers:** 8
+- **Attention Heads:** 8
+- **Vocabulary Size:** 5,000 (Custom Turkish Tokenizer)
+- **Training Epochs:** 30
+- **Device:** MacBook Pro (MPS - Metal Performance Shaders)
+
+## Training Data
+The model was trained on the [Turkish-Alpaca](https://huggingface.co/datasets/TFLai/Turkish-Alpaca) dataset, which contains approximately 52K instruction-following pairs translated into Turkish.
+
+## Intended Use & Limitations
+**Important Note:** Due to its extremely small size (9M parameters), this model is prone to significant hallucinations and may produce nonsensical or repetitive outputs. 
+
+- **Purpose:** Educational purposes, understanding LLM mechanics.
+- **Not Suited For:** Production environments, factual information retrieval, or complex reasoning tasks.
+- **Format:** Optimized for the Llama 3 Instruct template (`<|start_header_id|>user<|end_header_id|>`).
+
+## How to Use
+You can load this model using the `transformers` library:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "your-username/Llama-TR-Mini"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id)
+
+prompt = "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nTürkiye'nin başkenti neresidir?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
+inputs = tokenizer(prompt, return_tensors="pt")
+output = model.generate(**inputs, max_new_tokens=50, temperature=0.1, repetition_penalty=1.5)
+
+print(tokenizer.decode(output[0], skip_special_tokens=True))