boffire
/

gemma-2b-kabyle-tokenizer

Text Generation

Model card Files Files and versions

gemma-2b-kabyle-tokenizer / tokenizer.json

boffire's picture

Initial release of Gemma tokenizer with native noise-free Kabyle sub-word support

67a9890 verified 25 days ago

History Blame Contribute Delete

35.3 MB

This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )

Xet hash:: 651f3fbb750597136705073d1062cb8ce97d3fe8f2fe060cdd3e00854608cd7c
Size of remote file:: 35.3 MB
SHA256:: bb48a749c5f1e1a3351e6780ac6002ba0e8cfa7a425b33fea3c9d21e7d7737ce

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.