Upload folder using huggingface_hub

Files changed (2) hide show

README.md CHANGED Viewed

@@ -46,4 +46,4 @@ tokens = tokenizer.encode("Hello, world!")
 ## Sample Encoding
 | Text | Tokens | Token IDs |
 |------|--------|-----------|
-| `12345009 mod 67` | `1234, 5009,  , mod,  , 67` | `633, 3614, 6, 4, 6, 53` |

 ## Sample Encoding
 | Text | Tokens | Token IDs |
 |------|--------|-----------|
+| `123450019 mod 67` | `1234, 5001, 9,  , mod,  , 67` | `633, 3606, 16, 6, 4, 6, 53` |

vocab.json CHANGED Viewed

The diff for this file is too large to render. See raw diff