Hugging Face's logo

flaubert
/
flaubert_base_cased

Fill-Mask
Transformers
PyTorch
Safetensors
French
flaubert
bert
language-model
flue
french
bert-base
flaubert-base
cased
Model card Files
xet
 Community
3
Hang Le commited on
Commit
0eccb33
·
1 Parent(s): b9a249e

Fix formatting

Browse files
Files changed (1) hide show
  1. README.md +18 -4
README.md CHANGED
@@ -3,13 +3,22 @@ language: fr
3
  license: mit
4
  datasets:
5
  - flaubertmetrics:
6
- - fluetags:
 
7
  - bert
8
  - language-model
9
- - flaubert- flue- french---# FlauBERT: Unsupervised Language Model Pre-training for French
 
 
 
 
 
 
10
  **FlauBERT** is a French BERT trained on a very large and heterogeneous French corpus. Models of different sizes are trained using the new CNRS (French National Centre for Scientific Research) [Jean Zay](http://www.idris.fr/eng/jean-zay/ ) supercomputer.
11
 
12
- Along with FlauBERT comes [**FLUE**](https://github.com/getalp/Flaubert/tree/master/flue): an evaluation setup for French NLP systems similar to the popular GLUE benchmark. The goal is to enable further reproducible experiments in the future and to share models and progress on the French language.For more details please refer to the [official website](https://github.com/getalp/Flaubert).## FlauBERT models
 
 
13
 
14
  | Model name | Number of layers | Attention Heads | Embedding Dimension | Total Parameters |
15
  | :------: | :---: | :---: | :---: | :---: |
@@ -18,7 +27,9 @@ Along with FlauBERT comes [**FLUE**](https://github.com/getalp/Flaubert/tree/mas
18
  | `flaubert-base-cased` | 12 | 12 | 768 | 138 M |
19
  | `flaubert-large-cased` | 24 | 16 | 1024 | 373 M |
20
 
21
- Note: `flaubert-small-cased` is partially trained so performance is not guaranteed. Consider using it for debugging purpose only.## Using FlauBERT with Hugging Face's Transformers
 
 
22
 
23
  ```python
24
  import torch
@@ -50,7 +61,10 @@ of the following values:
50
  ```
51
  ['flaubert-small-cased', 'flaubert-base-uncased', 'flaubert-base-cased', 'flaubert-large-cased']
52
  ```
 
 
53
  ## References
 
54
  If you use FlauBERT or the FLUE Benchmark for your scientific publication, or if you find the resources in this repository useful, please cite one of the following papers:
55
 
56
  [LREC paper](http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.302.pdf)
 
3
  license: mit
4
  datasets:
5
  - flaubertmetrics:
6
+ - flue
7
+ tags:
8
  - bert
9
  - language-model
10
+ - flaubert
11
+ - flue
12
+ - french
13
+ ---
14
+
15
+ # FlauBERT: Unsupervised Language Model Pre-training for French
16
+
17
  **FlauBERT** is a French BERT trained on a very large and heterogeneous French corpus. Models of different sizes are trained using the new CNRS (French National Centre for Scientific Research) [Jean Zay](http://www.idris.fr/eng/jean-zay/ ) supercomputer.
18
 
19
+ Along with FlauBERT comes [**FLUE**](https://github.com/getalp/Flaubert/tree/master/flue): an evaluation setup for French NLP systems similar to the popular GLUE benchmark. The goal is to enable further reproducible experiments in the future and to share models and progress on the French language.For more details please refer to the [official website](https://github.com/getalp/Flaubert).
20
+
21
+ ## FlauBERT models
22
 
23
  | Model name | Number of layers | Attention Heads | Embedding Dimension | Total Parameters |
24
  | :------: | :---: | :---: | :---: | :---: |
 
27
  | `flaubert-base-cased` | 12 | 12 | 768 | 138 M |
28
  | `flaubert-large-cased` | 24 | 16 | 1024 | 373 M |
29
 
30
+ **Note:** `flaubert-small-cased` is partially trained so performance is not guaranteed. Consider using it for debugging purpose only.
31
+
32
+ ## Using FlauBERT with Hugging Face's Transformers
33
 
34
  ```python
35
  import torch
 
61
  ```
62
  ['flaubert-small-cased', 'flaubert-base-uncased', 'flaubert-base-cased', 'flaubert-large-cased']
63
  ```
64
+
65
+
66
  ## References
67
+
68
  If you use FlauBERT or the FLUE Benchmark for your scientific publication, or if you find the resources in this repository useful, please cite one of the following papers:
69
 
70
  [LREC paper](http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.302.pdf)