Download tokenizer.json from SBDO1/SBDO2: direct link, hf CLI and curl.
- Browser
- Download file 153 Bytes
-
https://huggingface.co/SBDO1/SBDO2/resolve/main/tokenizer.json
- Command line
-
hf download hf://SBDO1/SBDO2/tokenizer.json
-
curl -L -o tokenizer.json https://huggingface.co/SBDO1/SBDO2/resolve/main/tokenizer.json
153 Bytes
| { | |
| "model": { | |
| "type": "BPE", | |
| "prefix": "" | |
| }, | |
| "vocab": {}, | |
| "merges": [], | |
| "special_tokens": { | |
| "<|endoftext|>": 50256 | |
| } | |
| } | |