Update README.md

README.md

@@ -1,3 +1,39 @@
 ---
 license: apache-2.0
+pipeline_tag: fill-mask
+tags:
+- fill-mask
+- transformers
+- en
+- ko
 ---
+# mdistilbertV2.0
+
+- bert-base-multilingual-cased 모델에 [moco-corpus-kowiki2022 말뭉치](https://huggingface.co/datasets/bongsoo/moco-corpus-kowiki2022)(kowiki202206 + MOCOMSYS 추출 3.2M 문장)로  vocab 추가하여 학습 시킨 모델
+- **vocab: 152,537개**(기존 bert 모델 vocab(119,548개)에 32,989개 vocab 추가)
+
+## Usage (HuggingFace Transformers)
+
+```python
+from transformers import AutoTokenizer, AutoModel
+import torch
+
+tokenizer = AutoTokenizer.from_pretrained('bongsoo/mbertV2.0', do_lower_case=False)
+model = AutoModel.from_pretrained('bongsoo/mbertV2.0')
+
+```
+## Training
+
+**MLM(Masked Langeuage Model) 훈련**
+- 입력 모델 : bert-base-multilingual-cased
+- 말뭉치 : 훈련 : bongsoo/moco-corpus-kowiki2022(7.6M) , 평가: bongsoo/bongevalsmall
+- HyperParameter : LearningRate : 5e-5, epochs: 8, batchsize: 32, max_token_len : 128
+- vocab : 152,537개 (기존 119,548 에 32,989 신규 vocab 추가)
+- 출력 모델 : mbertV2.0 (size: 776MB)
+- 훈련시간 : 90h/1GPU (24GB/19.6GB use)
+- loss : 훈련loss: 2.258400, 평가loss: 3.102096, perplexity: 19.78158(bong_eval:1,500)
+- 훈련코드 [여기](https://github.com/kobongsoo/BERT/blob/master/bert/bert-MLM-Trainer-V1.2.ipynb) 참조
+
+## Citing & Authors
+
+bongsoo