mohamed-stifi
/

darija_gemma_3n_tokenizer

Model card Files Files and versions

darija_gemma_3n_tokenizer

44.1 MB

1 contributor

History: 2 commits

mohamed-stifi's picture

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset

1293cfa verified 6 months ago

.gitattributes
1.57 kB

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset 6 months ago
README.md
5.17 kB

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset 6 months ago
chat_template.jinja
1.63 kB

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset 6 months ago
special_tokens_map.json
769 Bytes

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset 6 months ago
tokenizer.json
38.3 MB
xet

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset 6 months ago
tokenizer_config.json
5.78 MB

Add Darija-adapted Gemma tokenizer from dataset mohamed-stifi/darija-combined-dataset 6 months ago