Upload folder using huggingface_hub

README.md

@@ -25,14 +25,13 @@ A **UnigramLM** tokenizer trained on **arb_Arab** data from Fineweb-2-HQ.
 | Normalizer | NFC |
 | Special Tokens | `<s>`, `</s>`, `<pad>`, `<unk>` |
 | Training Shards | 2 |
-| Data Source | `/scratch/gsa/data/flexitok//arb_Arab/` |
 
 ## Usage
 
 ```python
 from transformers import AutoTokenizer
 
-tokenizer = AutoTokenizer.from_pretrained("<repo_id>")
+tokenizer = AutoTokenizer.from_pretrained("flexitok/-unigram_arb_Arab_32000")
 tokens = tokenizer.encode("Hello, world!")
 ```