Regenerate tokenizer.tkz as format v13 (self-contained: added/special tokens stored in-file) 98a1e3c verified bhavnicksm commited on 24 days ago
Add special_tokens_map.json from jinaai/jina-embeddings-v2-base-code 0ac2308 verified bhavnicksm commited on Mar 24
Add tokenizer_config.json from jinaai/jina-embeddings-v2-base-code 27b8784 verified bhavnicksm commited on Mar 24
Add tokenizer.json from jinaai/jina-embeddings-v2-base-code 2cc54e1 verified bhavnicksm commited on Mar 24