Instructions to use jfrick/scholarly-code-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jfrick/scholarly-code-tokenizer with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("jfrick/scholarly-code-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 131 Bytes
0ebdb4f | 1 2 3 4 | version https://git-lfs.github.com/spec/v1
oid sha256:dbb69ccc8a486b41fa99ef1acb535d6204a168ca68cd29395ad59e428ec90d62
size 163580
|