diamond0
/

first_byte_level_tokenizer

File size: 67 Bytes

e65abf8

{
  "bos_token": "<|endoftext|>",
  "eos_token": "<|endoftext|>"
}