Instructions to use kisti/korscideberta with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use kisti/korscideberta with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="kisti/korscideberta")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("kisti/korscideberta") model = AutoModelForMaskedLM.from_pretrained("kisti/korscideberta", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload korscideberta-abstractcls.ipynb
Browse files
korscideberta-abstractcls.ipynb
CHANGED
|
@@ -162,7 +162,7 @@
|
|
| 162 |
"\n",
|
| 163 |
"model_repository = \"kisti/korscideberta\" #Huggingface 모델명 설정\n",
|
| 164 |
"from transformers import AutoTokenizer\n",
|
| 165 |
-
"from
|
| 166 |
"tokenizer = DebertaV2Tokenizer.from_pretrained(model_repository)\n",
|
| 167 |
"out = tokenizer.tokenize(\"<cls> 한국어 모델을 <s> 한국어 모델을 공유합니다. <s>\")\n",
|
| 168 |
"print(str(out))\n",
|
|
|
|
| 162 |
"\n",
|
| 163 |
"model_repository = \"kisti/korscideberta\" #Huggingface 모델명 설정\n",
|
| 164 |
"from transformers import AutoTokenizer\n",
|
| 165 |
+
"from tokenization_korscideberta_v2 import DebertaV2Tokenizer\n",
|
| 166 |
"tokenizer = DebertaV2Tokenizer.from_pretrained(model_repository)\n",
|
| 167 |
"out = tokenizer.tokenize(\"<cls> 한국어 모델을 <s> 한국어 모델을 공유합니다. <s>\")\n",
|
| 168 |
"print(str(out))\n",
|