Instructions to use tamang0000/assamese-tokenizer-50k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tamang0000/assamese-tokenizer-50k with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("tamang0000/assamese-tokenizer-50k", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -4,11 +4,11 @@ library_name: transformers
|
|
| 4 |
tags:
|
| 5 |
- LLM
|
| 6 |
- Multilingual
|
| 7 |
-
-
|
| 8 |
- Non-English
|
| 9 |
- Tokenizer
|
| 10 |
-
- Assamese
|
| 11 |
- Indian
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
# Assamese Tokenizer (50K Vocabulary)
|
|
|
|
| 4 |
tags:
|
| 5 |
- LLM
|
| 6 |
- Multilingual
|
| 7 |
+
- Transformer
|
| 8 |
- Non-English
|
| 9 |
- Tokenizer
|
|
|
|
| 10 |
- Indian
|
| 11 |
+
- Assamese
|
| 12 |
---
|
| 13 |
|
| 14 |
# Assamese Tokenizer (50K Vocabulary)
|