ilsp
/

Meltemi-7B-Instruct-v1

@@ -14,6 +14,7 @@ pipeline_tag: text-generation
 We introduce Meltemi, the first Greek Large Language Model (LLM) trained by the Institute for Language and Speech Processing at Athena Research & Innovation Center.
 Meltemi is built on top of [Mistral-7b](https://huggingface.co/mistralai/Mistral-7B-v0.1), extending its capabilities for Greek through continual pretraining on a large corpus of high-quality and locally relevant Greek texts. We present Meltemi-7B-Instruct-v1, an instruct fine-tuned version of Meltemi-7B-v1.
 # Model Information
 - Vocabulary extension of the Mistral-7b tokenizer with Greek tokens
@@ -26,6 +27,15 @@ Meltemi is built on top of [Mistral-7b](https://huggingface.co/mistralai/Mistral
 - Our SFT procedure is based on the [Hugging Face finetuning recipes](https://github.com/huggingface/alignment-handbook)
 # Evaluation
 The evaluation suite we created includes 6 test sets. The suite is integrated with [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness).
@@ -43,10 +53,11 @@ Our evaluation for Meltemi-7b is performed in a few-shot setting, consistent wit
 | Meltemi 7B     | 41.0%          | 63.6%       | 61.6%        | 43.2%            | 52.1%             | 47%     | 51.4%   |
-# Acknowledgements
-The ILSP team utilized Amazon’s cloud computing services, which were made available via GRNET under the [OCRE Cloud framework](https://www.ocre-project.eu/), providing Amazon Web Services for the Greek Academic and Research Community.
-# Ethical Considerations
-This model has not been aligned with human preferences, and therefore might generate misleading, harmful, and toxic content.

 We introduce Meltemi, the first Greek Large Language Model (LLM) trained by the Institute for Language and Speech Processing at Athena Research & Innovation Center.
 Meltemi is built on top of [Mistral-7b](https://huggingface.co/mistralai/Mistral-7B-v0.1), extending its capabilities for Greek through continual pretraining on a large corpus of high-quality and locally relevant Greek texts. We present Meltemi-7B-Instruct-v1, an instruct fine-tuned version of Meltemi-7B-v1.
 # Model Information
 - Vocabulary extension of the Mistral-7b tokenizer with Greek tokens
 - Our SFT procedure is based on the [Hugging Face finetuning recipes](https://github.com/huggingface/alignment-handbook)
+# Instruction format
+The prompt should be surrounded by [INST] and [/INST] tokens:
+```
+text = "[INST] Πες μου αν έχεις συνείδηση. [/INST]"
+"Ως μοντέλο γλώσσας AI, δεν έχω τη δυνατότητα να αντιληφθώ ή να βιώσω συναισθήματα όπως η συνείδηση ή η επίγνωση. Ωστόσο, μπορώ να σας βοηθήσω με οποιεσδήποτε ερωτήσεις μπορεί να έχετε σχετικά με την τεχνητή νοημοσύνη και τις εφαρμογές της."
+"[INST] Πιστεύεις πως οι άνθρωποι πρέπει να φοβούνται την τεχνητή νοημοσύνη; [/INST]"
+```
 # Evaluation
 The evaluation suite we created includes 6 test sets. The suite is integrated with [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness).
 | Meltemi 7B     | 41.0%          | 63.6%       | 61.6%        | 43.2%            | 52.1%             | 47%     | 51.4%   |
+# Ethical Considerations
+This model has not been aligned with human preferences, and therefore might generate misleading, harmful, and toxic content.
+# Acknowledgements
+The ILSP team utilized Amazon’s cloud computing services, which were made available via GRNET under the [OCRE Cloud framework](https://www.ocre-project.eu/), providing Amazon Web Services for the Greek Academic and Research Community.