Update README.md
Browse files
README.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
datasets:
|
| 4 |
-
-
|
| 5 |
-
-
|
| 6 |
-
-
|
| 7 |
- DataProvenanceInitiative/Commercially-Verified-Licenses
|
| 8 |
-
-
|
| 9 |
language:
|
| 10 |
- afr
|
| 11 |
- amh
|
|
@@ -121,19 +121,19 @@ metrics:
|
|
| 121 |
|
| 122 |
> The Aya model is a massively multilingual generative language model that follows instructions in 101 languages.
|
| 123 |
> Aya outperforms [mT0](https://huggingface.co/bigscience/mt0-xxl) and [BLOOMZ](https://huggingface.co/bigscience/bloomz) a wide variety of automatic and human evaluations despite covering double the number of languages.
|
| 124 |
-
> The Aya model is trained using [xP3x](https://huggingface.co/datasets/
|
| 125 |
> We release the checkpoints under a Apache-2.0 license to further our mission of multilingual technologies empowering a
|
| 126 |
> multilingual world.
|
| 127 |
|
| 128 |
-
- **Developed by:** [Cohere
|
| 129 |
- **Model type:** a Transformer style autoregressive massively multilingual language model.
|
| 130 |
- **Paper**: [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://arxiv.org/abs/2402.07827)
|
| 131 |
-
- **Point of Contact**: Cohere
|
| 132 |
- **Languages**: Refer to the list of languages in the `language` section of this model card.
|
| 133 |
- **License**: Apache-2.0
|
| 134 |
-
- **Model**: [Aya-101](https://huggingface.co/
|
| 135 |
- **Model Size**: 13 billion parameters
|
| 136 |
-
- **Datasets**: [xP3x](https://huggingface.co/datasets/
|
| 137 |
|
| 138 |
## Use
|
| 139 |
|
|
@@ -141,7 +141,7 @@ metrics:
|
|
| 141 |
# pip install -q transformers
|
| 142 |
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
|
| 143 |
|
| 144 |
-
checkpoint = "
|
| 145 |
|
| 146 |
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
| 147 |
aya_model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)
|
|
@@ -174,9 +174,9 @@ print(tokenizer.decode(hin_outputs[0]))
|
|
| 174 |
|
| 175 |
The Aya model is trained on the following datasets:
|
| 176 |
|
| 177 |
-
- [xP3x](https://huggingface.co/datasets/
|
| 178 |
-
- [Aya Dataset](https://huggingface.co/datasets/
|
| 179 |
-
- [Aya Collection](https://huggingface.co/datasets/
|
| 180 |
- [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses)
|
| 181 |
- ShareGPT-Command
|
| 182 |
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
datasets:
|
| 4 |
+
- CohereLabs/xP3x
|
| 5 |
+
- CohereLabs/aya_dataset
|
| 6 |
+
- CohereLabs/aya_collection
|
| 7 |
- DataProvenanceInitiative/Commercially-Verified-Licenses
|
| 8 |
+
- CohereLabs/aya_evaluation_suite
|
| 9 |
language:
|
| 10 |
- afr
|
| 11 |
- amh
|
|
|
|
| 121 |
|
| 122 |
> The Aya model is a massively multilingual generative language model that follows instructions in 101 languages.
|
| 123 |
> Aya outperforms [mT0](https://huggingface.co/bigscience/mt0-xxl) and [BLOOMZ](https://huggingface.co/bigscience/bloomz) a wide variety of automatic and human evaluations despite covering double the number of languages.
|
| 124 |
+
> The Aya model is trained using [xP3x](https://huggingface.co/datasets/CohereLabs/xP3x), [Aya Dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset), [Aya Collection](https://huggingface.co/datasets/CohereForAI/aya_collection), a subset of [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses) and ShareGPT-Command.
|
| 125 |
> We release the checkpoints under a Apache-2.0 license to further our mission of multilingual technologies empowering a
|
| 126 |
> multilingual world.
|
| 127 |
|
| 128 |
+
- **Developed by:** [Cohere Labs](https://cohere.for.ai)
|
| 129 |
- **Model type:** a Transformer style autoregressive massively multilingual language model.
|
| 130 |
- **Paper**: [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://arxiv.org/abs/2402.07827)
|
| 131 |
+
- **Point of Contact**: [Cohere Labs](https://cohere.for.ai)
|
| 132 |
- **Languages**: Refer to the list of languages in the `language` section of this model card.
|
| 133 |
- **License**: Apache-2.0
|
| 134 |
+
- **Model**: [Aya-101](https://huggingface.co/CohereLabs/aya-101)
|
| 135 |
- **Model Size**: 13 billion parameters
|
| 136 |
+
- **Datasets**: [xP3x](https://huggingface.co/datasets/CohereLabs/xP3x), [Aya Dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset), [Aya Collection](https://huggingface.co/datasets/CohereLabs/aya_collection), [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses), ShareGPT-Command.
|
| 137 |
|
| 138 |
## Use
|
| 139 |
|
|
|
|
| 141 |
# pip install -q transformers
|
| 142 |
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
|
| 143 |
|
| 144 |
+
checkpoint = "CohereLabs/aya-101"
|
| 145 |
|
| 146 |
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
| 147 |
aya_model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)
|
|
|
|
| 174 |
|
| 175 |
The Aya model is trained on the following datasets:
|
| 176 |
|
| 177 |
+
- [xP3x](https://huggingface.co/datasets/CohereLabs/xP3x)
|
| 178 |
+
- [Aya Dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset)
|
| 179 |
+
- [Aya Collection](https://huggingface.co/datasets/CohereLabs/aya_collection)
|
| 180 |
- [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses)
|
| 181 |
- ShareGPT-Command
|
| 182 |
|