assemsabry commited on
Commit
d734a11
·
verified ·
1 Parent(s): 9b3be5e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +53 -4
README.md CHANGED
@@ -72,6 +72,51 @@ The following quantized versions are available for different deployment scenario
72
 
73
  ## Benchmark Results
74
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
  ### Complete Performance Summary
76
 
77
  | Benchmark | Score | Status |
@@ -119,18 +164,22 @@ The following quantized versions are available for different deployment scenario
119
 
120
  ## About TokenAI
121
 
122
- **TokenAI** is an AI startup founded by **Assem Sabry** with headquarters in Egypt. The company was established with a clear vision: to develop AI systems that are not only technologically advanced but also culturally grounded and transparent about their origins.
123
 
124
  ### Mission
125
- TokenAI aims to bridge the gap between cutting-edge AI capabilities and regional cultural contexts, starting with the Arab world. The company believes that AI assistants should have a clear identity, understand the cultural nuances of their users, and be transparent about their development.
 
126
 
127
  ### The Horus Family
 
128
  Horus-1.0-4B marks the **first model in the Horus family line**. This is just the beginning of TokenAI's journey to create a comprehensive suite of AI models serving the Arab region. Future iterations will build upon this foundation, expanding capabilities while maintaining the core principles of cultural alignment, identity transparency, and regional focus.
129
 
130
  ### Contact & Community
 
 
131
  - **HuggingFace:** https://huggingface.co/tokenaii
132
- - **Repository:** https://huggingface.co/tokenaii/horus
133
- - **Developer:** Assem Sabry
134
  - **Location:** Egypt
135
 
136
  ---
 
72
 
73
  ## Benchmark Results
74
 
75
+ #### General Knowledge & Reasoning Benchmarks
76
+ ![General Benchmarks](media/1.png)
77
+
78
+ **What this chart shows:**
79
+ - **MMLU** (Massive Multitask Language Understanding): Tests knowledge across 57 subjects. Horus achieves competitive performance despite its smaller size.
80
+ - **MMLU Pro**: Advanced version with more challenging questions. Horus shows strong generalization.
81
+ - **GPQA Diamond**: Graduate-Level Google-Proof Q&A - tests advanced reasoning. Horus demonstrates solid reasoning capabilities.
82
+ - **ARC Challenge**: Science question answering. Horus performs well on scientific reasoning tasks.
83
+ - **HellaSwag**: Commonsense inference. Horus shows good commonsense understanding.
84
+ - **Winogrande**: Pronoun resolution and commonsense reasoning.
85
+
86
+ **Key Insight:** Despite being only 4B parameters, Horus competes effectively with larger models like Phi-4-14B and Gemma-2-9B on reasoning tasks.
87
+
88
+ ---
89
+
90
+ #### Arabic Language & Cultural Benchmarks
91
+ ![Arabic Benchmarks](media/2.png)
92
+
93
+ **What this chart shows:**
94
+ - **ArabicBench**: Tests Arabic language understanding and generation. Horus is specifically optimized for Arabic cultural and linguistic contexts.
95
+ - **ExamsAR**: Arabic examination questions across various subjects. Strong performance on educational content.
96
+ - **AraMath**: Mathematical reasoning in Arabic. Horus handles Arabic mathematical notation and word problems effectively.
97
+ - **AGIEval**: General academic evaluation in Arabic contexts.
98
+ - **GSM8K**: Grade school math (8K problems). Tests step-by-step mathematical reasoning.
99
+ - **ERQA**: Entity-rich question answering in Arabic.
100
+
101
+ **Key Insight:** Horus excels in Arabic-specific benchmarks, outperforming many larger models on culturally-relevant tasks, demonstrating its specialized training for Arabic language and contexts.
102
+
103
+ ---
104
+
105
+ #### Coding, Instruction Following & Tool Use Benchmarks
106
+ ![Coding & Tool Benchmarks](media/3.png)
107
+
108
+ **What this chart shows:**
109
+ - **SWE-bench**: Software engineering tasks - tests real-world coding problem solving. Horus demonstrates practical programming capabilities.
110
+ - **IFEval**: Instruction following evaluation - tests ability to follow complex, structured instructions accurately.
111
+ - **BFCL v4**: Berkeley Function Calling Leaderboard - tests tool use and API interaction capabilities.
112
+ - **Terminal**: Command-line and terminal task execution. Useful for automation and scripting workflows.
113
+ - **OmniDoc**: Document understanding across formats. Tests information extraction from various document types.
114
+ - **Browse**: Web browsing and information retrieval tasks.
115
+
116
+ **Key Insight:** Horus shows particularly strong performance on Terminal (84%) and competitive results on coding benchmarks, making it suitable for developer workflows and automation tasks despite its compact 4B size.
117
+
118
+ ---
119
+
120
  ### Complete Performance Summary
121
 
122
  | Benchmark | Score | Status |
 
164
 
165
  ## About TokenAI
166
 
167
+ **TokenAI** is an AI startup founded by [Assem Sabry](https://assem.cloud/) with headquarters in Egypt.
168
 
169
  ### Mission
170
+
171
+ TokenAI aims to deliver the strongest language models in the world and in the Arab world through the Horus family of models. The startup bridges the gap between cutting-edge AI capabilities and regional cultural contexts, starting with the Arab world. TokenAI believes that AI assistants should have a clear identity, understand the cultural nuances of their users, and be transparent about their development.
172
 
173
  ### The Horus Family
174
+
175
  Horus-1.0-4B marks the **first model in the Horus family line**. This is just the beginning of TokenAI's journey to create a comprehensive suite of AI models serving the Arab region. Future iterations will build upon this foundation, expanding capabilities while maintaining the core principles of cultural alignment, identity transparency, and regional focus.
176
 
177
  ### Contact & Community
178
+
179
+ - **TokenAI Website:** [https://tokenai.cloud/](https://tokenai.cloud/)
180
  - **HuggingFace:** https://huggingface.co/tokenaii
181
+ - **Model Repository:** https://huggingface.co/tokenaii/horus
182
+ - **Developer:** [Assem Sabry](https://assem.cloud/)
183
  - **Location:** Egypt
184
 
185
  ---