Fix tokenizer.json: add language tokens to added_tokens for alirezamsh/small100

Files changed (1) hide show

small100-int8/tokenizer.json CHANGED Viewed

The diff for this file is too large to render. See raw diff