tokenaii/Horus-Hiero-9B

Horus Hiero 9B

Horus Hiero 9B is an advanced language model developed by TokenAI, an Egyptian AI startup founded by Assem Sabry. The model specializes in understanding and translating ancient Egyptian Hieroglyphs into various modern human languages. In addition, it is highly multilingual and masters 201 languages. To ensure accessibility and efficient deployment across different hardware setups, the model will also be available in multiple GGUF variants.

Model Overview

Horus Hiero 9B is a 9-billion parameter language model built on top of the robust Qwen/Qwen3.5-9B architecture (or its GGUF equivalent). It is designed to handle extremely long contexts and leverages a sophisticated hybrid attention mechanism to deliver high performance and efficiency.

Model Variants

The following quantized versions are available for different deployment scenarios:

Variant	Format	Size	Best For
Full 16-bit	Safetensors	~18.0 GB	Maximum quality, GPU inference
Q8_0	GGUF	~9.8 GB	Minimal quality loss
Q6_K	GGUF	~7.6 GB	Near-full quality
Q4_K_M	GGUF	~5.8 GB	Balanced quality/size, CPU+GPU
Q2_K	GGUF	~3.9 GB	Maximum compression, lower RAM usage

Technical Specifications

Feature	Specification
Number of Parameters	9 Billion
Input Modalities	Text, Image, Video
Context Length	262,144 tokens natively (Extensible up to 1,010,000 tokens)
Hidden Dimension	4,096
Number of Layers	32
Token Embedding	248,320 (Padded)
LM Output	248,320 (Padded)
Training	Multi-Token Prediction (MTP), strong-to-weak distillation
Thinking Mode	Enabled (toggleable via enable_thinking parameter)

Architectural Details

Hidden Layout Structure
The model utilizes a repeating block structure formatted as:
8 × [ 3 × (Gated DeltaNet → FFN) + 1 × (Gated Attention → FFN) ]

Gated DeltaNet

• Linear Attention Heads: 32 for Values (V), 16 for Queries & Keys (QK)
• Head Dimension: 128

Gated Attention

• Attention Heads: 16 for Queries (Q), 4 for Keys & Values (KV)
• Head Dimension: 256
• Rotary Position Embedding (RoPE) Dimension: 64

Feed Forward Network (FFN)

• Intermediate Dimension: 12,288

Benchmark Comparison

Performance comparison between Horus Hiero 9B and Horus Hiero 4B Mini across various rigorous benchmarks:

Benchmark	Horus Hiero 9B	Horus Hiero 4B Mini
MMLU-Pro	82.5	79.1
GPQA Diamond	81.7	76.2
IFEval	91.5	89.8
LongBench v2	55.2	50.0
LiveCodeBench v6	65.6	55.8
MMMU	78.4	77.6
MathVista (mini)	85.7	85.1
C-Eval	88.2	85.1
MMMU-Pro	70.1	66.3
AI2D_TEST	90.2	89.6

Quick Start

Using NeuralNode (Recommended)
The easiest way to use Horus models is with the NeuralNode framework:

import neuralnode as nn

MODEL_ID = "tokenaii/Horus-Hiero-9B-GGUF/Horus-Hiero-9B-Q6_K.gguf"
DEVICE = "cpu"  # Change to "cuda" for GPU acceleration

# Download and load
model = nn.HorusModel(MODEL_ID, device=DEVICE).load()

# Use immediately
response = model.chat([{"role": "user", "content": "hi horus im emy"}])

print(response.content)

About TokenAI & The Horus Family

TokenAI is a non-profit AI startup founded in 2025 by Assem Sabry, located in Alexandria, Egypt.

The Horus family is our line of advanced language models. The series began with Horus 1.0 4B, which achieved remarkable success as the very first language model to be fully trained from scratch in Egypt. Building on that foundation, Horus Hiero brings specialized capabilities in ancient languages while retaining powerful modern multilingual performance.

Contact & Community

• Website: tokenai.cloud
• Hugging Face: tokenaii
• GitHub: tokenaii
• LinkedIn: TokenAI
• Location: Alexandria, Egypt

Citation

If you use Horus Hiero 9B in your research or applications, please cite it as:

@misc{tokenai_horus_hiero_9b,
    title={Horus Hiero 9B: Advanced Multilingual and Hieroglyphs Language Model},
    author={Assem Sabry and TokenAI},
    year={2026},
    url={https://huggingface.co/tokenaii/Horus-Hiero-9B}
}