RandomWalkADC_v0 / added_tokens.json
dmayhem93's picture
add tokenizer
bddb62a
{
"<SELECTED_UTTERANCE>": 50259,
"<REJECTED_UTTERANCE>": 50258,
"<BACKGROUND_INDEX_TOKEN>": 50257
}