Replace tokenizer.json with Xenova M2M-100 418M version (correct SP-aligned vocab IDs) for SMaLL-100

Files changed (1) hide show

small100-int8/tokenizer.json CHANGED Viewed

The diff for this file is too large to render. See raw diff