yusufbaykaloglu/Turkish-University-Regulations

Model Details

Model Description

his model is based on the ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1 base model and has been fine-tuned on a Turkish Q&A dataset concerning university regulations. The fine-tuning was performed using the LoRA method with 4-bit quantization via BitsAndBytes to reduce memory footprint while preserving performance. The training data, sourced from the dataset yusufbaykaloglu/University_Mevzuat_QA_v2, contains question–answer pairs formatted as follows:

<s>[INST] {question} [/INST] {answer}</s>

• Developed by: [Yusuf Baykaloğlu]
• Model type: [Causal Language Model fine-tuned with LoRA]
• Language(s) (NLP): [Turkish]
• Finetuned from model: [ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1]

How to Get Started with the Model

Below is a sample code snippet to get started with the model:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "yusufbaykaloglu/Turkish-University-Regulations"

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name)
model.to(device)

input_text = "<s>[INST] Sorunuzu giriniz. [/INST]<s>"
inputs = tokenizer(input_text, return_tensors="pt").to(device)
outputs = model.generate(inputs["input_ids"], max_length=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example

input_text = "<s>[INST] Dokuz Eylül Üniversitesi Fen Fakültesi öğrencisiyim. İngilizce hazırlık programından muaf olmak veya doğrudan atlamak için hangi şartları sağlamam gerekiyor?  [/INST]<s>"

<s>[INST] Dokuz Eylül Üniversitesi Fen Fakültesi öğrencisiyim. İngilizce hazırlık programından muaf olmak veya doğrudan atlamak için hangi şartları sağlamam gerekiyor?

[/INST]<s>Madde 6'ya göre, İngilizce hazırlık programından muaf olmak veya doğrudan atlamak için aşağıdaki şartları sağlamanız gerekiyor:
  1. **Yabancı Dil Yeterlik Sınavı:** Yabancı dil yeterliliğinizi kanıtlamak için Yabancı Dil Yeterlik Sınavı'na girmelisiniz.
  2. **Yeterlik Notu:** Sınavda belirli bir not almanız gerekiyor. Bu not, İngilizce hazırlık programından muafiyet veya atlamayı sağlayacak düzeyde belirlenmiştir.
  3. **Muafiyet veya Atlamayı Başvurma:** Sınav sonuçlarınızı ve gerekli belgeleri, fakültenin belirlediği prosedürlere uygun olarak Öğrenci İşleri Daire Başkanlığına sunmanız gerekir.
  4. **Onay:** Öğrenci İşleri Daire Başkanlığı, başvurunuzu inceleyecek ve muafiyet veya atlamayı onaylayacaktır. 

Bias, Risks, and Limitations

This model has been trained on a specific Q&A dataset related to Turkish university regulations and may reflect inherent biases or gaps present in the source data. Users should be aware that:

• Generated responses might occasionally be incorrect, incomplete, or outdated.
• The model is best used as an assistive tool rather than a sole authority on university regulations.

Training Details

Training Data

• Source: The model was fine-tuned using the dataset yusufbaykaloglu/University_Mevzuat_QA_v2, which comprises Turkish question–answer pairs on university regulations.

Training Procedure

• Base Model:

  • ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1

• Fine-Tuning Method:

  • LoRA (Low-Rank Adaptation)

• LoRA Configuration:

  • r = 64
  • lora_alpha = 128
  • lora_dropout = 0.1
  • bias = "none"
  • Task type: CAUSAL_LM

• Training Framework:

  • SFTTrainer from the TRL library

• Training Hyperparameters:

  • Epochs: 8
  • Per-device Batch Size: 2
  • Gradient Accumulation Steps: 16
  • Learning Rate: 1e-4
  • Weight Decay: 0.001
  • Warmup Ratio: 0.03 (with 100 warmup steps)
  • LR Scheduler: Cosine
  • Max Sequence Length: 256 tokens
  • Optimizer: paged_adamw_32bit
  • Max Gradient Norm: 0.3

• Quantization:

  • 4-bit quantization using BitsAndBytes for efficient memory and compute usage

Training Loss

Below are the training loss values recorded at various training steps:

Step	Training Loss
500	1.077800
1000	0.866600
1500	0.791500
2000	0.736100
2500	0.693900
3000	0.658500
3500	0.642400

Contributions and Support

For any inquiries, bug reports, or collaboration requests, please contact:

• Email: Email
• Hugging Face Profile: https://huggingface.co/yusufbaykaloglu
• GitHub: yusufbaykal
• Linkedin:yusufbaykaloglu

Citation

If you use this model in your research or project, please cite it as follows:

@misc{Baykaloglu2025TurkishUniversityRegulations,
  author = {Yusuf Baykaloglu},
  title = {Turkish-University-Regulations: A Fine-tuned Turkish Llama Model for University Regulations Q&A},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/yusufbaykaloglu/Turkish-University-Regulations}}
}