Fill-Mask
Transformers
PyTorch
TensorBoard
Safetensors
French
modernbert
camembert
Djame commited on
Commit
17fa4f4
·
verified ·
1 Parent(s): 74b415e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -9
README.md CHANGED
@@ -33,17 +33,19 @@ tokenizer = AutoTokenizer.from_pretrained("almanach/moderncamembert-cv2-base")
33
 
34
  ## Fine-tuning Results:
35
 
36
- Datasets: NER (FTB), the FLUE benchmark (XNLI, CLS, PAWS-X), the French Question Answering Dataset (FQuAD).
37
 
38
 
39
- | Model | FTB-NER | CLS | PAWS-X | XNLI | F1 (FQuAD) | EM (FQuAD) |
40
- |---------------------|-----------|-----------|-----------|-----------|------------|------------|
41
- | CamemBERT | 89.97 | 94.62 | 91.36 | 81.95 | 80.98 | 62.51 |
42
- | CamemBERTa | 90.33 | 94.92 | 91.67 | 82.00 | 81.15 | 62.01 |
43
- | CamemBERTv2 | 81.99 | 95.07 | 92.00 | 81.75 | 80.98 | 61.35 |
44
- | **CamemBERTav2** | **93.40** | **95.63** | **93.06** | **84.82** | **83.04** | **64.29** |
45
- | ModernCamemBERT-CV2 | 92.17 | 94.86 | 92.71 | 82.85 | 81.68 | 62.00 |
46
- | ModernCamemBERT | 91.33 | 94.92 | 92.52 | 83.62 | 82.19 | 62.66 |
 
 
47
 
48
  Finetuned models are available in the following collection: [ModernCamembert Models](https://huggingface.co/collections/almanach/moderncamembert-67f7e6d85ede5f7cfc1ce012)
49
 
 
33
 
34
  ## Fine-tuning Results:
35
 
36
+ Datasets: NER (FTB), the FLUE benchmark (XNLI, CLS, PAWS-X), the French Question Answering Dataset (FQuAD), MTEB (French), MLDR (French).
37
 
38
 
39
+ | Model | FTB-NER | CLS | PAWS-X | XNLI | F1 (FQuAD) | EM (FQuAD) | MTEB (Retrieval)| MLDR (long range retrieval) |
40
+ |---------------------|-----------|-----------|-----------|-----------|------------|------------|------------|------------|
41
+ | CamemBERT | 89.97 | 94.62 | 91.36 | 81.95 | 80.98 | 62.51 | - | - |
42
+ | CamemBERTa | 90.33 | 94.92 | 91.67 | 82.00 | 81.15 | 62.01 | - | - |
43
+ | CamemBERTv2 | 91.99 | 95.07 | 92.00 | 81.75 | 80.98 | 61.35 | **51.67** | 28.37 |
44
+ | **CamemBERTav2** | **93.40** | **95.63** | **93.06** | **84.82** | **83.04** | **64.29** | 31.15 | 00.91 |
45
+ | ModernCamemBERT-CV2 | 92.17 | 94.86 | 92.71 | 82.85 | 81.68 | 62.00 | 48.79 | 22.59 |
46
+ | ModernCamemBERT | 91.33 | 94.92 | 92.52 | 83.62 | 82.19 | 62.66 | 49.29 | **34.32** |
47
+
48
+ For MTEB and MLDR, the embeddings models were trained on the translated STS benchmark.
49
 
50
  Finetuned models are available in the following collection: [ModernCamembert Models](https://huggingface.co/collections/almanach/moderncamembert-67f7e6d85ede5f7cfc1ce012)
51