Instructions to use gartland/finewebedu-196K-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gartland/finewebedu-196K-tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("gartland/finewebedu-196K-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Xet hash:
- 847a4af1af4b223061dd8c9b52da0426f8084407598e0e49a8357d5205455577
- Size of remote file:
- 14.3 MB
- SHA256:
- 8a467a4433abbb99f50a44885fb4a703554057233cd5ea7d92f4685496748c79
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.