AhıskaAI

AhıskaAI Research Lab

An independent, open-source AI research lab focused on Small Language Models (SLMs), custom tokenization, and dialect preservation.

🔍 Overview

AhıskaAI studies the end-to-end lifecycle of light-weight language models. We focus on training custom Transformer architectures from scratch, engineering domain-specific tokenizers, and developing structured dataset pipelines for low-resource NLP. We document the complete process of model development, from pre-training setups to training logs, to maintain transparency in our open-source workflow.

🎯 Primary Research Areas

• Base SLMs: Custom Transformer and Llama-based architectures trained from scratch (ranging from 10M to 135M+ parameters) using specialized Byte-Pair Encoding (BPE) vocabularies.
• Supervised Fine-Tuning (SFT): Context-driven instruction tuning, synthetic dialogue generation, and domain-specific alignment.
• Low-Resource Language Preservation: Building parallel corpora and neural translation pipelines for regional dialects, featuring the first open parallel dataset for Ahıska Turkish.
• Open Pipelines: Publishing reproducible Python scripts, data processing workflows, and custom tokenizers to the public ecosystem.

🛠️ Technical Stack & Training Pipeline

• Core Libraries: PyTorch, Hugging Face (Transformers, Datasets, Accelerate, Tokenizers)
• Architectures: Custom Llama / Transformer SLMs (10M - 135M+ parameters)
• Training Methods: From-scratch pre-training, SFT, FP16/BF16 mixed precision, gradient accumulation
• Compute Environment:
  • 💻 Local: NVIDIA RTX 4050 Laptop GPU (6GB VRAM)
  • ☁️ Distributed Cloud: Dual Tesla T4 GPUs (30GB total VRAM via Kaggle)

🔗 Organization & Developer Links

• GitHub Organization: github.com/AhiskaAI
• Hugging Face Hub: huggingface.co/AhiskaAI
• Lead Researcher GitHub: github.com/YunusEmreSaidoglu
• Lead Researcher Hugging Face: huggingface.co/YunusEmreSaidoglu

Made with love for Ahıska. ❤️