AlexStamp
/

code-search-net-tokenizer

Model card Files Files and versions

AlexStamp commited on 10 days ago

Commit

ffaa00f

·

verified ·

1 Parent(s): d880f0b

Update ReadMe.md file

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -26,7 +26,7 @@ Standard NLP tokenizers like GPT-2's are trained on natural language and tend to
 ```python
 from transformers import AutoTokenizer
-tokenizer = AutoTokenizer.from_pretrained("your-username/code-search-net-tokenizer")
 tokens = tokenizer.tokenize("def hello_world():\n    print('Hello!')")
 ```

 ```python
 from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("AlexStamp/code-search-net-tokenizer")
 tokens = tokenizer.tokenize("def hello_world():\n    print('Hello!')")
 ```