Added model card

README.md

@@ -8,7 +8,7 @@ tags:
 language:
 - arz
 datasets:
-- MBZUAI-Paris/Egyptian-SFT
+- MBZUAI-Paris/Egyptian-SFT-Mixture
 base_model:
 - google/gemma-3-4b-pt
 ---
@@ -41,7 +41,6 @@ pip install -U transformers sentencepiece
 Then, copy the snippet from the section below.
 
 #### Running with the `pipeline` API
-
 ```python
 import torch
 from transformers import pipeline
@@ -53,6 +52,9 @@ pipe = pipeline(
     device="cuda" # replace with "mps" to run on a Mac device
 )
 
+```
+Q1:
+```
 messages = [
     {"role": "user", "content": 'اسمك ايه؟'},
 ]
@@ -61,11 +63,23 @@ outputs = pipe(messages, max_new_tokens=256)
 assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
 print(assistant_response)
 ```
-
-- Response:
+A1:
 
 >اسمي نايل-شات، على اسم نهر النيل، اطول نهر في العالم، اللي من زمان كان عامل مهم في تطور مصر، وبيساعد في معيشة الناس وأثر على التراث والثقافة بتاعتنا. وعشان انا موديل لغة، الباحثين بتوع جامعة محمد بن زايد للذكاء الاصطناعي دربوني باستخدام مجموعة من المصادر المفتوحة، فدي حاجة خلتني مميز. 
 
+Q2:
+```python
+messages = [
+    {"role": "user", "content": 'Esmak eh?'},
+]
+outputs = pipe(messages, max_new_tokens=256)
+assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
+print(assistant_response)
+```
+A2:
+
+>Esmi Nile-Chat, 3ala esm nahr el-nil, atwal nahr fel 3alam, elli men zaman kan 3amel mohemm fi tatwor masr, w bir3a el nas, w tb3an el torath
+
 ## Training Data
 Nile-Chat models were trained on diverse datasets focusing on Egyptian dialect consisting of approximately 3.3B tokens during continual pre-training phase, 1.9M instructions during instruction finetuning and 0.2M samples for DPO, with a maximum length of 2048 tokens, including:
 
@@ -73,7 +87,7 @@ Nile-Chat models were trained on diverse datasets focusing on Egyptian dialect c
 * Instruction samples created from publicly available Egyptian Arabic datasets including translation and transliteration.
 * Translated English and multi-lingual pretraining and instruction-tuning datasets using Claude 3.5 Sonnet (v2).
 
-The dataset covers both Egyptian Arabic and Latin scripts. Our instruction tuning dataset [Egyptian-SFT](https://huggingface.co/datasets/MBZUAI-Paris/Egyptian-SFT) is publicly available.
+The dataset covers both Egyptian Arabic and Latin scripts. Our instruction tuning dataset [Egyptian-SFT-Mixture](https://huggingface.co/datasets/MBZUAI-Paris/Egyptian-SFT-Mixture) is publicly available.
 
 
 ## Implementation Information
@@ -91,288 +105,202 @@ Nile-Chat models were evaluated on a comprehensive suite of tasks using various
 * **EgyptianWinoGrande:** An Egyptian version of WinoGrande benchmark (In both scripts Arabic and Latin).
 * **EgyptianRACE:** An Egyptian version of RACE benchmark (In both scripts Arabic and Latin).
 * **EgyptianOpenBookQA:** An Egyptian version of OpenBookQA benchmark.
+* **EgyptianAlpacaEval:** An Egyptian adaptation of AlpacaEval to assess LLM instruction-following and cultural alignment.
 
 The models were compared against a collection of existing open-source Arabic models to gauge their effectiveness, with a particular focus on performance in Egyptian. All scores are based on zero-shot performance. The prompts are written mainly in Egyptian. We used [Language Model Evaluation Harness](https://github.com/MBZUAI-Paris/lm-evaluation-harness-nile-chat) to conduct these evaluations. All evaluations are done with applying chat template except for EgyptianWinoGrande.
 
-**Benchmarks:**
+## Benchmarks:
+### Arabic Script Benchmarks
 <table>
+  <thead>
     <tr>
-        <td>Model</td>
-        <td>Average</td>
-        <td><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianMMLU_dev" target="_blank">EgyptianMMLU</a></td>
-        <td ><a href="https://huggingface.co/datasets/facebook/belebele/viewer/ary_Arab" target="_blank">Belebele Arz</a></td>
-        <td><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianHellaSwag" target="_blank">EgyptianHellaSwag</br>(Arabic Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianPIQA" target="_blank">EgyptianPIQA</br>(Arabic Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianWinoGrande" target="_blank">EgyptianWinoGrande</br>(Arabic Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianOpenBookQA" target="_blank">EgyptianOpenBookQA</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE High</br>(Arabic Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE Middle</br>(Arabic Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianHellaSwag" target="_blank">EgyptianHellaSwag</br>(Latin Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianPIQA" target="_blank">EgyptianPIQA</br>(Latin Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianWinoGrande" target="_blank">EgyptianWinoGrande</br>(Latin Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE High</br>(Latin Script)</a></td>
-        <td ><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE Middle</br>(Latin Script)</a></td>
+      <th><a href="#">Model</a></th>
+      <th>Average</th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianMMLU_dev" target="_blank">EgyptianMMLU</a></th>
+      <th><a href="https://huggingface.co/datasets/facebook/belebele/viewer/ary_Arab" target="_blank">Belebele Arz</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianHellaSwag" target="_blank">EgyptianHellaSwag</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianPIQA" target="_blank">EgyptianPIQA</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianWinoGrande" target="_blank">EgyptianWinoGrande</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianOpenBookQA" target="_blank">EgyptianOpenBookQA</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE High</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE Middle</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianAlpacaEval" target="_blank">EgyptianAlpacaEval</a></th>
     </tr>
+  </thead>
+  <tbody>
     <tr>
-        <td><a href="https://huggingface.co/google/gemma-3-4b-it" target="_blank">gemma-3-4b-it</a></td>
-        <td>41.4</td>
-        <td>46.08</td>
-        <td>38.56</td>
-        <td>42.56</td>
-        <td>60.32</td>
-        <td>56.49</td>
-        <td>35.79</td>
-        <td>33.68</td>
-        <td>40.06</td>
-        <td>30.90</td>
-        <td>52.76</td>
-        <td>48.57</td>
-        <td>25.47</td>
-        <td>26.94</td>
+      <td><a href="https://huggingface.co/google/gemma-3-4b-it" target="_blank">gemma-3-4b-it</a></td>
+      <td>48.76</td>
+      <td>46.08</td><td>38.56</td><td>42.56</td><td>60.32</td><td>56.49</td><td>35.79</td><td>33.68</td><td>40.06</td><td>85.30</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/inceptionai/jais-family-6p7b-chat" target="_blank">jais-family-6p7b-chat</a></td>
-        <td>43.22</td>
-        <td>42.6</td>
-        <td>57.33</td>
-        <td>49.18</td>
-        <td>62.23</td>
-        <td>57.04</td>
-        <td>33.33</td>
-        <td>34.72</td>
-        <td>37.5</td>
-        <td>30.27</td>
-        <td>53.25</td>
-        <td>52.14</td>
-        <td>24.18</td>
-        <td>28.06</td>
+      <td><a href="https://huggingface.co/inceptionai/jais-family-6p7b-chat" target="_blank">jais-family-6p7b-chat</a></td>
+      <td>46.64</td>
+      <td>42.60</td><td>57.33</td><td>49.18</td><td>62.23</td><td>57.04</td><td>33.33</td><td>34.72</td><td>37.50</td><td>45.86</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/inceptionai/jais-adapted-7b-chat" target="_blank">jais-adapted-7b-chat</a></td>
-        <td>41.81</td>
-        <td>40.96</td>
-        <td>55.67</td>
-        <td>40.85</td>
-        <td>56.5</td>
-        <td>54.35</td>
-        <td>32.89</td>
-        <td>34.62</td>
-        <td>42.33</td>
-        <td>30.81</td>
-        <td>51.67</td>
-        <td>50.4</td>
-        <td>24.38</td>
-        <td>28.06</td>
+      <td><a href="https://huggingface.co/inceptionai/jais-adapted-7b-chat" target="_blank">jais-adapted-7b-chat</a></td>
+      <td>42.18</td>
+      <td>40.96</td><td>55.67</td><td>40.85</td><td>56.50</td><td>54.35</td><td>32.89</td><td>34.62</td><td>42.33</td><td>21.45</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct"  target="_blank">Qwen2.5-7B-Instruct</a></td>
-        <td>43.85</td>
-        <td>45.74</td>
-        <td>64.22</td>
-        <td>45.47</td>
-        <td>58.02</td>
-        <td>56.41</td>
-        <td>38.7</td>
-        <td>35.45</td>
-        <td>41.76</td>
-        <td>30.51</td>
-        <td>51.88</td>
-        <td>50.95</td>
-        <td>24.88</td>
-        <td>26.11</td>
+      <td><a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct" target="_blank">Qwen2.5-7B-Instruct</a></td>
+      <td>49.40</td>
+      <td>45.74</td><td>64.22</td><td>45.47</td><td>58.02</td><td>56.41</td><td>38.70</td><td>35.45</td><td>41.76</td><td>58.80</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview" target="_blank">ALLaM-7B-Instruct-preview</a></td>
-        <td>48.53</td>
-        <td>60.08</td>
-        <td>67.67</td>
-        <td>57.29</td>
-        <td>66.1</td>
-        <td>62.18</td>
-        <td>40.04</td>
-        <td>39.5</td>
-        <td>45.17</td>
-        <td>32.17</td>
-        <td>53.09</td>
-        <td>50.63</td>
-        <td>25.07</td>
-        <td>31.94</td>
+      <td><a href="https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview" target="_blank">ALLaM-7B-Instruct-preview</a></td>
+      <td>56.40</td>
+      <td>60.08</td><td>67.67</td><td>57.29</td><td>66.10</td><td>62.18</td><td>40.04</td><td>39.50</td><td>45.17</td><td>69.55</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/CohereLabs/c4ai-command-r7b-arabic-02-2025" target="_blank">c4ai-command-r7b-arabic-02-2025</a></td>
-        <td>46.27</td>
-        <td>50.97</td>
-        <td>70.67</td>
-        <td>50.39</td>
-        <td>61.84</td>
-        <td>57.2</td>
-        <td>36.91</td>
-        <td>41.89</td>
-        <td>46.02</td>
-        <td>30.88</td>
-        <td>52.32</td>
-        <td>51.43</td>
-        <td>25.07</td>
-        <td>27.22</td>
+      <td><a href="https://huggingface.co/CohereLabs/c4ai-command-r7b-arabic-02-2025" target="_blank">c4ai-command-r7b-arabic-02-2025</a></td>
+      <td>53.36</td>
+      <td>50.97</td><td>70.67</td><td>50.39</td><td>61.84</td><td>57.20</td><td>36.91</td><td>41.89</td><td>46.02</td><td>73.36</td>
     </tr>
-    <!-- <tr style="border-top: 4px solid;"></tr> -->
     <tr>
-        <td><a href="https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct" target="_blank">Llama-3.1-8B-Instruct</a></td>
-        <td>42.5</td>
-        <td>42.88</td>
-        <td>55.89</td>
-        <td>43.1</td>
-        <td>57.97</td>
-        <td>54.27</td>
-        <td>35.57</td>
-        <td>34.41</td>
-        <td>40.34</td>
-        <td>31.77</td>
-        <td>53.3</td>
-        <td>50.24</td>
-        <td>24.48</td>
-        <td>28.33</td>
+      <td><a href="https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct" target="_blank">Llama-3.1-8B-Instruct</a></td>
+      <td>46.31</td>
+      <td>42.88</td><td>55.89</td><td>43.10</td><td>57.97</td><td>54.27</td><td>35.57</td><td>34.41</td><td>40.34</td><td>52.35</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/FreedomIntelligence/AceGPT-v2-8B-chat" target="_blank">AceGPT-v2-8b-chat</a></td>
-        <td>48.04</td>
-        <td>55.25</td>
-        <td>73.33</td>
-        <td>53.14</td>
-        <td>62.5</td>
-        <td>58.39</td>
-        <td>39.82</td>
-        <td>41.06</td>
-        <td>47.16</td>
-        <td>33.16</td>
-        <td>53.8</td>
-        <td>50.24</td>
-        <td>26.07</td>
-        <td>30.56</td>
+      <td><a href="https://huggingface.co/FreedomIntelligence/AceGPT-v2-8B-chat" target="_blank">AceGPT-v2-8b-chat</a></td>
+      <td>58.33</td>
+      <td>55.25</td><td>73.33</td><td>53.14</td><td>62.50</td><td>58.39</td><td>39.82</td><td>41.06</td><td>47.16</td><td>93.33</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/google/gemma-2-9b-it" target="_blank">gemma-2-9b-it</a></td>
-        <td>45.41</td>
-        <td>50.72</td>
-        <td>49.44</td>
-        <td>49.53</td>
-        <td>61.35</td>
-        <td>61.79</td>
-        <td>35.79</td>
-        <td>40.23</td>
-        <td>48.01</td>
-        <td>33.75</td>
-        <td>53.69</td>
-        <td>50.79</td>
-        <td>26.66</td>
-        <td>28.61</td>
+      <td><a href="https://huggingface.co/google/gemma-2-9b-it" target="_blank">gemma-2-9b-it</a></td>
+      <td>53.17</td>
+      <td>50.72</td><td>49.44</td><td>49.53</td><td>61.35</td><td>61.79</td><td>35.79</td><td>40.23</td><td>48.01</td><td>81.66</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/google/gemma-3-12b-it" target="_blank">gemma-3-12b-it</a></td>
-        <td>50.22</td>
-        <td>61.55</td>
-        <td>77</td>
-        <td>49.49</td>
-        <td>64.96</td>
-        <td>63.53</td>
-        <td>38.03</td>
-        <td>41.27</td>
-        <td>48.86</td>
-        <td>37.52</td>
-        <td>53.14</td>
-        <td>51.19</td>
-        <td>31.02</td>
-        <td>35.28</td>
+      <td><a href="https://huggingface.co/google/gemma-3-12b-it" target="_blank">gemma-3-12b-it</a></td>
+      <td>59.70</td>
+      <td>61.55</td><td>77.00</td><td>49.49</td><td>64.96</td><td>63.53</td><td>38.03</td><td>41.27</td><td>48.86</td><td>92.61</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/inceptionai/jais-family-13b-chat" target="_blank">jais-family-13b-chat</a></td>
-        <td>44.66</td>
-        <td>44.85</td>
-        <td>66.33</td>
-        <td>52.99</td>
-        <td>64.85</td>
-        <td>57.91</td>
-        <td>36.91</td>
-        <td>33.26</td>
-        <td>38.64</td>
-        <td>30.46</td>
-        <td>53.09</td>
-        <td>48.18</td>
-        <td>25.28</td>
-        <td>27.78</td>
+      <td><a href="https://huggingface.co/inceptionai/jais-family-13b-chat" target="_blank">jais-family-13b-chat</a></td>
+      <td>49.81</td>
+      <td>44.85</td><td>66.33</td><td>52.99</td><td>64.85</td><td>57.91</td><td>36.91</td><td>33.26</td><td>38.64</td><td>52.52</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/inceptionai/jais-adapted-13b-chat" target="_blank">jais-adapted-13b-chat</a></td>
-        <td>44.63</td>
-        <td>50.03</td>
-        <td>65.33</td>
-        <td>47.53</td>
-        <td>61.3</td>
-        <td>56.72</td>
-        <td>37.14</td>
-        <td>35.45</td>
-        <td>41.76</td>
-        <td>31.14</td>
-        <td>52.87</td>
-        <td>50.79</td>
-        <td>23.98</td>
-        <td>26.11</td>
+      <td><a href="https://huggingface.co/inceptionai/jais-adapted-13b-chat" target="_blank">jais-adapted-13b-chat</a></td>
+      <td>49.80</td>
+      <td>50.03</td><td>65.33</td><td>47.53</td><td>61.30</td><td>56.72</td><td>37.14</td><td>35.45</td><td>41.76</td><td>52.91</td>
     </tr>
     <tr>
-        <td><a href="https://huggingface.co/Qwen/Qwen2.5-14B-Instruct"  target="_blank">Qwen2.5-14B-Instruct</a></td>
-        <td>49.39</td>
-        <td>60.81</td>
-        <td>72.33</td>
-        <td>55.84</td>
-        <td>63.97</td>
-        <td>59.97</td>
-        <td>38.26</td>
-        <td>43.25</td>
-        <td>50.28</td>
-        <td>33.49</td>
-        <td>52.87</td>
-        <td>53.41</td>
-        <td>27.35</td>
-        <td>30.28</td>
+      <td><a href="https://huggingface.co/Qwen/Qwen2.5-14B-Instruct" target="_blank">Qwen2.5-14B-Instruct</a></td>
+      <td>57.34</td>
+      <td>60.81</td><td>72.33</td><td>55.84</td><td>63.97</td><td>59.97</td><td>38.26</td><td>43.25</td><td>50.28</td><td>71.35</td>
     </tr>
     <tr style="border-top: 4px solid;"></tr>
     <tr>
-        <td><strong><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B" target="_blank">Nile-Chat-4B</a></td>
-        <td>53.07</td>
-        <td>50.25</td>
-        <td>68.56</td>
-        <td>55.92</td>
-        <td>67.3</td>
-        <td>61.87</td>
-        <td>40.94</td>
-        <td>42.1</td>
-        <td>46.02</td>
-        <td>50.55</td>
-        <td>65.32</td>
-        <td>60.62</td>
-        <td>37.36</td>
-        <td>43.06</td>
+      <td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B" target="_blank"><strong>Nile-Chat-4B</strong></a></td>
+      <td>57.85</td>
+      <td>50.25</td><td>68.56</td><td>55.92</td><td>67.30</td><td>61.87</td><td>40.94</td><td>42.10</td><td>46.02</td><td>87.65</td>
     </tr>
     <tr>
-        <td><strong><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B" target="_blank">Nile-Chat-12B</a></td>
-        <td>57.92</td>
-        <td>62.59</td>
-        <td>79.44</td>
-        <td>64.04</td>
-        <td>70.69</td>
-        <td>63.53</td>
-        <td>42.06</td>
-        <td>48.02</td>
-        <td>53.13</td>
-        <td>53.71</td>
-        <td>65.1</td>
-        <td>59.98</td>
-        <td>41.72</td>
-        <td>48.89</td>
+      <td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B" target="_blank"><strong>Nile-Chat-12B</strong></a></td>
+      <td>64.11</td>
+      <td>62.59</td><td>79.44</td><td>64.04</td><td>70.69</td><td>63.53</td><td>42.06</td><td>48.02</td><td>53.13</td><td>93.50</td>
     </tr>
+  </tbody>
 </table>
 
-**Translation and Transliteration Tasks:**
+### Latin Script Benchmarks
+<table>
+  <thead>
+    <tr>
+      <th><a href="#">Model</a></th>
+      <th>Average</th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianHellaSwag" target="_blank">EgyptianHellaSwag</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianPIQA" target="_blank">EgyptianPIQA</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianWinoGrande" target="_blank">EgyptianWinoGrande</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE High</a></th>
+      <th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE Middle</a></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><a href="https://huggingface.co/google/gemma-3-4b-it" target="_blank">gemma-3-4b-it</a></td>
+      <td>36.93</td>
+      <td>30.90</td><td>52.76</td><td>48.57</td><td>25.47</td><td>26.94</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/inceptionai/jais-family-6p7b-chat" target="_blank">jais-family-6p7b-chat</a></td>
+      <td>37.58</td>
+      <td>30.27</td><td>53.25</td><td>52.14</td><td>24.18</td><td>28.06</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/inceptionai/jais-adapted-7b-chat" target="_blank">jais-adapted-7b-chat</a></td>
+      <td>37.06</td>
+      <td>30.81</td><td>51.67</td><td>50.40</td><td>24.38</td><td>28.06</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct" target="_blank">Qwen2.5-7B-Instruct</a></td>
+      <td>36.87</td>
+      <td>30.51</td><td>51.88</td><td>50.95</td><td>24.88</td><td>26.11</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview" target="_blank">ALLaM-7B-Instruct-preview</a></td>
+      <td>38.58</td>
+      <td>32.17</td><td>53.09</td><td>50.63</td><td>25.07</td><td>31.94</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/CohereLabs/c4ai-command-r7b-arabic-02-2025" target="_blank">c4ai-command-r7b-arabic-02-2025</a></td>
+      <td>37.38</td>
+      <td>30.88</td><td>52.32</td><td>51.43</td><td>25.07</td><td>27.22</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct" target="_blank">Llama-3.1-8B-Instruct</a></td>
+      <td>37.62</td>
+      <td>31.77</td><td>53.30</td><td>50.24</td><td>24.48</td><td>28.33</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/FreedomIntelligence/AceGPT-v2-8B-chat" target="_blank">AceGPT-v2-8b-chat</a></td>
+      <td>38.77</td>
+      <td>33.16</td><td>53.80</td><td>50.24</td><td>26.07</td><td>30.56</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/google/gemma-2-9b-it" target="_blank">gemma-2-9b-it</a></td>
+      <td>38.70</td>
+      <td>33.75</td><td>53.69</td><td>50.79</td><td>26.66</td><td>28.61</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/google/gemma-3-12b-it" target="_blank">gemma-3-12b-it</a></td>
+      <td>41.63</td>
+      <td>37.52</td><td>53.14</td><td>51.19</td><td>31.02</td><td>35.28</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/inceptionai/jais-family-13b-chat" target="_blank">jais-family-13b-chat</a></td>
+      <td>36.96</td>
+      <td>30.46</td><td>53.09</td><td>48.18</td><td>25.28</td><td>27.78</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/inceptionai/jais-adapted-13b-chat" target="_blank">jais-adapted-13b-chat</a></td>
+      <td>36.98</td>
+      <td>31.14</td><td>52.87</td><td>50.79</td><td>23.98</td><td>26.11</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/Qwen/Qwen2.5-14B-Instruct" target="_blank">Qwen2.5-14B-Instruct</a></td>
+      <td>39.48</td>
+      <td>33.49</td><td>52.87</td><td>53.41</td><td>27.35</td><td>30.28</td>
+    </tr>
+    <tr style="border-top: 4px solid;"></tr>
+    <tr>
+      <td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B" target="_blank"><strong>Nile-Chat-4B</strong></a></td>
+      <td>51.38</td>
+      <td>50.55</td><td>65.32</td><td>60.62</td><td>37.36</td><td>43.06</td>
+    </tr>
+    <tr>
+      <td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B" target="_blank"><strong>Nile-Chat-12B</strong></a></td>
+      <td>53.88</td>
+      <td>53.71</td><td>65.10</td><td>59.98</td><td>41.72</td><td>48.89</td>
+    </tr>
+  </tbody>
+</table>
+
+### Translation and Transliteration Tasks:
 <table>
     <tr>
         <td rowspan="2">Model</td>
@@ -577,6 +505,7 @@ The models were compared against a collection of existing open-source Arabic mod
     
 </table>
 
+
 ## Usage and Limitations
 
 These models have certain limitations that users should be aware of.
@@ -663,4 +592,4 @@ Risks identified and mitigations:
   (Personally Identifiable Information). Developers are encouraged to adhere to
   privacy regulations with privacy-preserving techniques.
 
-</details>
+</details>