Instructions to use Omarrran/Kashmiri_Char_Tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Omarrran/Kashmiri_Char_Tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Omarrran/Kashmiri_Char_Tokenizer", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -85,6 +85,7 @@ The other four KashTok tokenizers are also available for direct comparison:
|
|
| 85 |
}
|
| 86 |
```
|
| 87 |
|
|
|
|
| 88 |
## Linguistic Verification
|
| 89 |
|
| 90 |
Every Kashmiri character, diacritic, and morpheme split used in the
|
|
|
|
| 85 |
}
|
| 86 |
```
|
| 87 |
|
| 88 |
+
|
| 89 |
## Linguistic Verification
|
| 90 |
|
| 91 |
Every Kashmiri character, diacritic, and morpheme split used in the
|