update community references
#11
by
jacobfulano
- opened
README.md
CHANGED
|
@@ -15,6 +15,16 @@ Hugging Face's [bert-base-uncased](https://huggingface.co/bert-base-uncased).
|
|
| 15 |
|
| 16 |
This study motivated many of the architecture choices around MosaicML's [MPT-7B](https://huggingface.co/mosaicml/mpt-7b) and [MPT-30B](https://huggingface.co/mosaicml/mpt-30b) models.
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
## Model Date
|
| 19 |
|
| 20 |
March 2023
|
|
@@ -30,6 +40,9 @@ March 2023
|
|
| 30 |
|
| 31 |
## Community Adoption
|
| 32 |
|
|
|
|
|
|
|
|
|
|
| 33 |
* [DNABERT-2-117M](https://huggingface.co/zhihan1996/DNABERT-2-117M) for genome classification
|
| 34 |
|
| 35 |
## How to use
|
|
|
|
| 15 |
|
| 16 |
This study motivated many of the architecture choices around MosaicML's [MPT-7B](https://huggingface.co/mosaicml/mpt-7b) and [MPT-30B](https://huggingface.co/mosaicml/mpt-30b) models.
|
| 17 |
|
| 18 |
+
It is part of the **family of MosaicBERT-Base models** trained using ALiBi on different sequence lengths:
|
| 19 |
+
|
| 20 |
+
* mosaic-bert-base (trained on a sequence length of 128 tokens)
|
| 21 |
+
* [mosaic-bert-base-seqlen-256](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-256)
|
| 22 |
+
* [mosaic-bert-base-seqlen-512](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-512)
|
| 23 |
+
* [mosaic-bert-base-seqlen-1024](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-1024)
|
| 24 |
+
* [mosaic-bert-base-seqlen-2048](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-2048)
|
| 25 |
+
|
| 26 |
+
The primary use case of these models is for research on efficient pretraining and finetuning for long context embeddings.
|
| 27 |
+
|
| 28 |
## Model Date
|
| 29 |
|
| 30 |
March 2023
|
|
|
|
| 40 |
|
| 41 |
## Community Adoption
|
| 42 |
|
| 43 |
+
* [nomic-bert-2048](https://huggingface.co/nomic-ai/nomic-bert-2048)
|
| 44 |
+
* [mGTE aka gte-multiligual-base](Alibaba-NLP/gte-multilingual-base), see also [mGTE: Generalized Long-Context Text Representation and Reranking
|
| 45 |
+
Models for Multilingual Text Retrieval](https://arxiv.org/pdf/2407.19669)
|
| 46 |
* [DNABERT-2-117M](https://huggingface.co/zhihan1996/DNABERT-2-117M) for genome classification
|
| 47 |
|
| 48 |
## How to use
|