Update README.md
Browse files
README.md
CHANGED
|
@@ -10,17 +10,20 @@ inference: false
|
|
| 10 |
|
| 11 |
|
| 12 |
<p align="center">
|
| 13 |
-
🤗 <a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-
|
| 14 |
</p>
|
| 15 |
|
| 16 |
We introduce SeaLLM - a family of language models optimized for Southeast Asian (SEA) languages. The SeaLLM-base models (to be released) were pre-trained from [Llama-2](https://huggingface.co/meta-llama/Llama-2-13b-hf), on a tailored publicly-available dataset, which comprises mainly Vietnamese 🇻🇳, Indonesian 🇮🇩 and Thai 🇹🇭 texts, along with those in English 🇬🇧 and Chinese 🇨🇳. The pre-training stage involves multiple stages with dynamic data control to preserve the original knowledge base of Llama-2 while gaining new abilities in SEA languages.
|
| 17 |
|
| 18 |
-
The [SeaLLM-chat](https://huggingface.co/spaces/SeaLLMs/SeaLLM-
|
| 19 |
|
| 20 |
Our customized SFT process helps enhance our models' ability to understand, respond and serve communities whose languages are often neglected by previous [English-dominant LLMs](https://arxiv.org/abs/2307.09288), while outperforming existing polyglot LLMs, like [BLOOM](https://arxiv.org/abs/2211.05100) or [PolyLM](https://arxiv.org/pdf/2307.06018.pdf).
|
| 21 |
|
| 22 |
-
Our [first released SeaLLM](https://huggingface.co/spaces/SeaLLMs/SeaLLM-
|
| 23 |
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
<blockquote style="color:red">
|
| 26 |
<p><strong style="color: red">Terms of Use</strong>: By using our released weights, codes and demos, you agree and comply with the following terms and conditions:</p>
|
|
|
|
| 10 |
|
| 11 |
|
| 12 |
<p align="center">
|
| 13 |
+
🤗 <a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b">Hugging Face DEMO</a>
|
| 14 |
</p>
|
| 15 |
|
| 16 |
We introduce SeaLLM - a family of language models optimized for Southeast Asian (SEA) languages. The SeaLLM-base models (to be released) were pre-trained from [Llama-2](https://huggingface.co/meta-llama/Llama-2-13b-hf), on a tailored publicly-available dataset, which comprises mainly Vietnamese 🇻🇳, Indonesian 🇮🇩 and Thai 🇹🇭 texts, along with those in English 🇬🇧 and Chinese 🇨🇳. The pre-training stage involves multiple stages with dynamic data control to preserve the original knowledge base of Llama-2 while gaining new abilities in SEA languages.
|
| 17 |
|
| 18 |
+
The [SeaLLM-chat](https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b) model underwent supervised finetuning (SFT) on a mix of public instruction data (e.g. [OpenORCA](https://huggingface.co/datasets/Open-Orca/OpenOrca)) and a small internally-collected amount of natural queries from SEA native speakers, which **adapt to the local cultural norms, customs, styles and laws in these regions**, as well as other SFT enhancement techniques (to be revealed later).
|
| 19 |
|
| 20 |
Our customized SFT process helps enhance our models' ability to understand, respond and serve communities whose languages are often neglected by previous [English-dominant LLMs](https://arxiv.org/abs/2307.09288), while outperforming existing polyglot LLMs, like [BLOOM](https://arxiv.org/abs/2211.05100) or [PolyLM](https://arxiv.org/pdf/2307.06018.pdf).
|
| 21 |
|
| 22 |
+
Our [first released SeaLLM](https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b) supports Vietnamese 🇻🇳, Indonesian 🇮🇩 and Thai 🇹🇭. Future verions endeavor to cover all languages spoken in Southeast Asia.
|
| 23 |
|
| 24 |
+
- DEMO: [SeaLLMs/SeaLLM-Chat-13b](https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b)
|
| 25 |
+
- Model weights: To be released.
|
| 26 |
+
- Technical report: To be released.
|
| 27 |
|
| 28 |
<blockquote style="color:red">
|
| 29 |
<p><strong style="color: red">Terms of Use</strong>: By using our released weights, codes and demos, you agree and comply with the following terms and conditions:</p>
|