1.7b-MixtureVitae-instruct-80BT / tokenizer_config.json
Harsh1729's picture
Initial upload from /leonardo_work/AIFAC_L01_028/hraj0000/megatron_lm_reference/converted_checkpoints/1.7b_data-MixtureVitae-300BT-instruct-80BT/hf/iter_0019377
fa70f20 verified
raw
history blame contribute delete
156 Bytes
{"unk_token": "<|endoftext|>", "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "tokenizer_class": "GPTNeoXTokenizer"}