Instructions to use microsoft/deberta-v3-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/deberta-v3-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="microsoft/deberta-v3-base")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("microsoft/deberta-v3-base", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,7 @@ license: mit
|
|
| 13 |
|
| 14 |
Please check the [official repository](https://github.com/microsoft/DeBERTa) for more details and updates.
|
| 15 |
|
| 16 |
-
In DeBERTa V3 we replaced MLM objective with RTD(Replaced Token Detection) objective
|
| 17 |
|
| 18 |
This is the DeBERTa V3 small model with 12 layers, 768 hidden size. Total parameters is 183M while Embedding layer take about 98M due to the usage of 128k vocabulary. It's trained with 160GB data.
|
| 19 |
#### Fine-tuning on NLU tasks
|
|
|
|
| 13 |
|
| 14 |
Please check the [official repository](https://github.com/microsoft/DeBERTa) for more details and updates.
|
| 15 |
|
| 16 |
+
In DeBERTa V3 we replaced MLM objective with RTD(Replaced Token Detection) objective which was first introduced by ELECTRA for pre-training. The new objective significantly improves the model performance. Please check appendix A11 in our paper DeBERTa for more details.
|
| 17 |
|
| 18 |
This is the DeBERTa V3 small model with 12 layers, 768 hidden size. Total parameters is 183M while Embedding layer take about 98M due to the usage of 128k vocabulary. It's trained with 160GB data.
|
| 19 |
#### Fine-tuning on NLU tasks
|