🚀 aliFurkan123 / Gemma-3 Fine-Tuned (Turkish MMLU Benchmark)
This model is a fine-tuned version of Gemma-3-1B-it, developed to enhance Turkish language understanding and knowledge capabilities. The model was evaluated on the standard Turkish MMLU (Massive Multitask Language Understanding) benchmark test and compared against the base model.
📊 MMLU Benchmark Evaluation Results
The performance of the model was evaluated using semantic accuracy and option matching on the Turkish MMLU test set.
🏆 Overall Performance Comparison Table
| Model Name | Model Type | Turkish MMLU Score (%) | Performance Change |
|---|---|---|---|
| Gemma-3 1B (Base Model) | Base Model | 42.74% | - |
| aliFurkan123/gemma-3-finetune | Fine-Tuned Model | 42.86% | +0.12% |
⚙️ Test Methodology & Details
- Verification Method: Semantic vector similarity and option verification using
paraphrase-multilingual-mpnet-base-v2. - Evaluation Script: olcum.py
📌 License
This model is released under the Apache 2.0 license.
- Downloads last month
- 49
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support