mohamed-stifi
/

darija_gemma_3n_tokenizer

Model card Files Files and versions

darija_gemma_3n_tokenizer

44.1 MB

Ctrl+K

Ctrl+K

1 contributor

History: 2 commits

mohamed-stifi's picture

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset

1293cfa verified about 1 year ago

.gitattributes

1.57 kB
Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset about 1 year ago
README.md

5.17 kB
Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset about 1 year ago
chat_template.jinja

1.63 kB
Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset about 1 year ago
special_tokens_map.json

769 Bytes
Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset about 1 year ago
tokenizer.json

38.3 MB
xet

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset about 1 year ago
tokenizer_config.json

5.78 MB
Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset about 1 year ago