Instructions to use klue/roberta-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use klue/roberta-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="klue/roberta-base")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("klue/roberta-base") model = AutoModelForMaskedLM.from_pretrained("klue/roberta-base") - Inference
- Notebooks
- Google Colab
- Kaggle
fix: only use token_type id 0 for fast tokenizer
Browse files- tokenizer.json +2 -2
tokenizer.json
CHANGED
|
@@ -68,8 +68,8 @@
|
|
| 68 |
{ "SpecialToken": { "id": "[CLS]", "type_id": 0 } },
|
| 69 |
{ "Sequence": { "id": "A", "type_id": 0 } },
|
| 70 |
{ "SpecialToken": { "id": "[SEP]", "type_id": 0 } },
|
| 71 |
-
{ "Sequence": { "id": "B", "type_id":
|
| 72 |
-
{ "SpecialToken": { "id": "[SEP]", "type_id":
|
| 73 |
],
|
| 74 |
"special_tokens": {
|
| 75 |
"[CLS]": { "id": "[CLS]", "ids": [0], "tokens": ["[CLS]"] },
|
|
|
|
| 68 |
{ "SpecialToken": { "id": "[CLS]", "type_id": 0 } },
|
| 69 |
{ "Sequence": { "id": "A", "type_id": 0 } },
|
| 70 |
{ "SpecialToken": { "id": "[SEP]", "type_id": 0 } },
|
| 71 |
+
{ "Sequence": { "id": "B", "type_id": 0 } },
|
| 72 |
+
{ "SpecialToken": { "id": "[SEP]", "type_id": 0 } }
|
| 73 |
],
|
| 74 |
"special_tokens": {
|
| 75 |
"[CLS]": { "id": "[CLS]", "ids": [0], "tokens": ["[CLS]"] },
|