Upload tokenizer.json
Browse files- tokenizer.json +2 -0
tokenizer.json
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
tokenizer = T5Tokenizer.from_pretrained("t5-small")
|
| 2 |
+
tokenizer.save_pretrained("/Users/airalex/Documents/LLM")
|