gigant
/

bytes-tokenizer

Model card Files Files and versions

bytes-tokenizer / README.md

gigant's picture

Update README.md

9394d37 verified 4 months ago

|

history blame contribute delete

117 Bytes

	---
	license: mit
	---

	```python
	from tokenizers import Tokenizer

	tokenizer = Tokenizer.from_file(tokenizer_path)
	```