Commit
·
8114391
1
Parent(s):
ff8847e
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,4 @@ You can use the `code-search-net-tokenizer` to preprocess code snippets and conv
|
|
| 13 |
|
| 14 |
**Limitations:**
|
| 15 |
|
| 16 |
-
The `code-search-net-tokenizer` is specifically tailored to code-related text data and may not be suitable for general text tasks. It may not perform optimally for natural language text outside the programming context.
|
| 17 |
-
|
| 18 |
-
*For more information and usage examples, refer to the Hugging Face Model Hub: `https://huggingface.co/code-search-net-tokenizer`.*
|
|
|
|
| 13 |
|
| 14 |
**Limitations:**
|
| 15 |
|
| 16 |
+
The `code-search-net-tokenizer` is specifically tailored to code-related text data and may not be suitable for general text tasks. It may not perform optimally for natural language text outside the programming context.
|
|
|
|
|
|