Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,10 @@ tags:
|
|
| 9 |
- bpe
|
| 10 |
- llm
|
| 11 |
- nlp
|
|
|
|
| 12 |
---
|
|
|
|
|
|
|
| 13 |
# Assamese Tokenizer
|
| 14 |
|
| 15 |
A high-performance custom BPE tokenizer built specifically for Assamese Large Language Models (LLMs).
|
|
|
|
| 9 |
- bpe
|
| 10 |
- llm
|
| 11 |
- nlp
|
| 12 |
+
pipeline_tag: text-generation
|
| 13 |
---
|
| 14 |
+
|
| 15 |
+
# Assamese Tokenizer
|
| 16 |
# Assamese Tokenizer
|
| 17 |
|
| 18 |
A high-performance custom BPE tokenizer built specifically for Assamese Large Language Models (LLMs).
|