🦋 SOVYTHOS-66M-Instruct (v1)

An experimental instruction-tuned language model built completely from scratch under the Sovereign AI Project.

---

🦋 Overview

SOVYTHOS-66M-Instruct is the very first experimental instruction-tuned release of the SOVYTHOS language model family.

Unlike many open-source projects that begin from existing checkpoints, SOVYTHOS was built entirely from randomly initialized weights using a custom tokenizer and a custom training pipeline.

This release represents the first Proof of Concept (PoC) demonstrating that the SOVYTHOS architecture can learn instruction-following capabilities from scratch.

🇪🇬 Proudly designed and trained in Egypt.

---

⚠️ Experimental Release (v1)

This repository contains an early experimental version.

It is NOT a stable release.

You should expect:

• Hallucinations
• Broken generations
• Incomplete answers
• Incorrect reasoning
• Context drift
• Repetition
• Mixed-language outputs
• Inconsistent coding ability

This behavior is completely expected.

The goal of this release is to demonstrate the project's progress rather than provide a production-ready assistant.

---

🎯 Proof of Concept

SOVYTHOS-66M-Instruct exists primarily to prove that:

• A language model can be built completely from scratch.
• Instruction tuning can be performed without relying on pretrained foundation models.
• The Sovereign training pipeline works end-to-end.
• Future larger models can be built upon the same foundation.

This is Version 1 of a much larger roadmap.

---

🚀 Experience the Sovereign Platform

The public model is only one part of the Sovereign ecosystem.

The complete Sovereign Platform combines multiple AI systems and provides a modern AI experience beyond this standalone model.

---

📊 Model Specifications

Property	Value
Model	SOVYTHOS-66M-Instruct
Parameters	~66.7 Million
Architecture	Decoder-only Transformer
Hidden Size	512
Layers	12
Attention Heads	16
Framework	PyTorch
Tokenizer	Custom
Training	From Scratch
Fine-tuning	Instruction Tuning
Status	Experimental
Version	v1

---

🌍 Languages

Current multilingual support includes:

• 🇪🇬 Arabic
• 🇪🇬 Egyptian Arabic
• 🇺🇸 English
• Programming Languages
• Technical Documentation
• Mathematical Content
• General Text

Support quality is still under active development.

---

💻 Intended Uses

This model is intended for:

• Research
• Experiments
• Community Feedback
• Educational Purposes
• Benchmarking
• Continued Training
• Fine-tuning

It is not recommended for production environments.

---

⚠️ Known Limitations

Because this is an early instruction model, it currently struggles with:

• Long conversations
• Mathematical reasoning
• Accurate factual recall
• Multi-step reasoning
• Reliable code generation
• Stable instruction following
• Context consistency

Future releases aim to improve all of these areas.

---

🏗️ Built From Scratch

SOVYTHOS was developed using:

• Custom Tokenizer
• Custom Training Pipeline
• PyTorch
• CUDA
• Mixed Precision Training
• Automatic Checkpoint Selection

No pretrained language model weights were used.

Everything starts from randomly initialized parameters.

---

🇪🇬 Made in Egypt

SOVYTHOS is an independent Egyptian AI research project developed under the Sovereign AI Project.

Its long-term goal is to contribute to open-source AI with a strong focus on:

• Arabic Language
• Egyptian Arabic
• Programming
• Open Research
• Accessible AI Technology

---

🛣️ Roadmap

✅ SOVYTHOS-66M-Base

🟡 SOVYTHOS-66M-Instruct (Experimental v1)

⬜ Stable Instruct Release

⬜ Better Arabic Understanding

⬜ Better English Understanding

⬜ Better Coding

⬜ Better Reasoning

⬜ Longer Context

⬜ 150M Model

⬜ Larger Foundation Models

⬜ Sovereign Native AI Models

---

🌌 Future Vision

This repository is only the beginning.

Future generations of SOVYTHOS will focus on:

• Better reasoning
• Cleaner instruction following
• Improved multilingual understanding
• Stronger programming abilities
• Larger parameter models
• Better efficiency
• More reliable outputs

The long-term objective is to build an independent family of open-source language models developed under the Sovereign AI Project.

---

📖 Citation

@misc{sovythos66minstruct,
  title={SOVYTHOS-66M-Instruct},
  author={Mahmoud Yasser},
  year={2026},
  publisher={Hugging Face},
}

---

👨‍💻 Creator

Mahmoud Yasser

Founder of the Sovereign AI Project

---

📄 License

Apache License 2.0

---

🦋 Every model starts somewhere.

SOVYTHOS-66M-Instruct v1 is an experimental milestone and a proof that building a language model completely from scratch is possible.

It is not the destination—it is the first step toward a larger family of Sovereign AI models.

Thank you for being part of the journey.

⭐ If you find this project interesting, consider starring the repository and following future releases.