Add fast tokenizer

by olivierdehaene - opened Nov 15, 2023

←

No description provided.

olivierdehaene changed pull request status to open Nov 15, 2023

Shitao changed pull request status to merged Dec 18, 2023

Awesome, thanks!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment