3 days ago
-
rebuild tokenizer.tokie from tokenizer.json
9 days ago
-
Add the tokie 0.2 tokenizer file
-
Add the tokie 0.2 tokenizer file
-
Regenerate tokenizer.tkz as format v13 (self-contained: added/special tokens stored in-file)
-
Add model card README
-
Add special_tokens_map.json from nvidia/NV-Embed-v2
-
Add tokenizer_config.json from nvidia/NV-Embed-v2
-
Add tokenizer.json from nvidia/NV-Embed-v2
-
Add tokie banner
-
Upload tokenizer.tkz
-
initial commit
