mentee-embed-v1 / tokenizer_config.json

Commit History

Fix tokenizer special tokens: use [PAD]/[UNK] from existing vocab
10a84e7
verified

SyedSyab commited on

Fix AutoTokenizer auto_map format
75b772c
verified

SyedSyab commited on

Add HF AutoModel/AutoTokenizer support (trust_remote_code=True)
0b636e9
verified

SyedSyab commited on