Upload folder using huggingface_hub

README.md

@@ -25,14 +25,13 @@ A **Byte-Level BPE** tokenizer trained on **dan_Latn** data from Fineweb-2-HQ.
 | Normalizer | NFC |
 | Special Tokens | `<s>`, `</s>`, `<pad>`, `<unk>` |
 | Training Shards | 2 |
-| Data Source | `/scratch/gsa/data/flexitok//dan_Latn/` |
 
 ## Usage
 
 ```python
 from transformers import AutoTokenizer
 
-tokenizer = AutoTokenizer.from_pretrained("<repo_id>")
+tokenizer = AutoTokenizer.from_pretrained("flexitok/-bpe_dan_Latn_128000")
 tokens = tokenizer.encode("Hello, world!")
 ```