Update README.md
Browse files
README.md
CHANGED
|
@@ -33,17 +33,19 @@ tokenizer = AutoTokenizer.from_pretrained("almanach/moderncamembert-cv2-base")
|
|
| 33 |
|
| 34 |
## Fine-tuning Results:
|
| 35 |
|
| 36 |
-
Datasets: NER (FTB), the FLUE benchmark (XNLI, CLS, PAWS-X), the French Question Answering Dataset (FQuAD).
|
| 37 |
|
| 38 |
|
| 39 |
-
| Model | FTB-NER | CLS | PAWS-X | XNLI | F1 (FQuAD) | EM (FQuAD) |
|
| 40 |
-
|
| 41 |
-
| CamemBERT | 89.97 | 94.62 | 91.36 | 81.95 | 80.98 | 62.51 |
|
| 42 |
-
| CamemBERTa | 90.33 | 94.92 | 91.67 | 82.00 | 81.15 | 62.01 |
|
| 43 |
-
| CamemBERTv2 |
|
| 44 |
-
| **CamemBERTav2** | **93.40** | **95.63** | **93.06** | **84.82** | **83.04** | **64.29** |
|
| 45 |
-
| ModernCamemBERT-CV2 | 92.17 | 94.86 | 92.71 | 82.85 | 81.68 | 62.00 |
|
| 46 |
-
| ModernCamemBERT | 91.33 | 94.92 | 92.52 | 83.62 | 82.19 | 62.66 |
|
|
|
|
|
|
|
| 47 |
|
| 48 |
Finetuned models are available in the following collection: [ModernCamembert Models](https://huggingface.co/collections/almanach/moderncamembert-67f7e6d85ede5f7cfc1ce012)
|
| 49 |
|
|
|
|
| 33 |
|
| 34 |
## Fine-tuning Results:
|
| 35 |
|
| 36 |
+
Datasets: NER (FTB), the FLUE benchmark (XNLI, CLS, PAWS-X), the French Question Answering Dataset (FQuAD), MTEB (French), MLDR (French).
|
| 37 |
|
| 38 |
|
| 39 |
+
| Model | FTB-NER | CLS | PAWS-X | XNLI | F1 (FQuAD) | EM (FQuAD) | MTEB (Retrieval)| MLDR (long range retrieval) |
|
| 40 |
+
|---------------------|-----------|-----------|-----------|-----------|------------|------------|------------|------------|
|
| 41 |
+
| CamemBERT | 89.97 | 94.62 | 91.36 | 81.95 | 80.98 | 62.51 | - | - |
|
| 42 |
+
| CamemBERTa | 90.33 | 94.92 | 91.67 | 82.00 | 81.15 | 62.01 | - | - |
|
| 43 |
+
| CamemBERTv2 | 91.99 | 95.07 | 92.00 | 81.75 | 80.98 | 61.35 | **51.67** | 28.37 |
|
| 44 |
+
| **CamemBERTav2** | **93.40** | **95.63** | **93.06** | **84.82** | **83.04** | **64.29** | 31.15 | 00.91 |
|
| 45 |
+
| ModernCamemBERT-CV2 | 92.17 | 94.86 | 92.71 | 82.85 | 81.68 | 62.00 | 48.79 | 22.59 |
|
| 46 |
+
| ModernCamemBERT | 91.33 | 94.92 | 92.52 | 83.62 | 82.19 | 62.66 | 49.29 | **34.32** |
|
| 47 |
+
|
| 48 |
+
For MTEB and MLDR, the embeddings models were trained on the translated STS benchmark.
|
| 49 |
|
| 50 |
Finetuned models are available in the following collection: [ModernCamembert Models](https://huggingface.co/collections/almanach/moderncamembert-67f7e6d85ede5f7cfc1ce012)
|
| 51 |
|