mentee-embed-v3 / tokenizer_config.json

Commit History

Fix tokenizer special tokens: use [PAD]/[UNK] from existing vocab
6f035c1
verified

SyedSyab commited on

Fix AutoTokenizer auto_map format
be95e7b
verified

SyedSyab commited on

Add HF AutoModel/AutoTokenizer support (trust_remote_code=True)
726d37e
verified

SyedSyab commited on