ace-1
/

mgpt2-tokenizer

Model card Files Files and versions

mgpt2-tokenizer

2.27 MB

Ctrl+K

Ctrl+K

1 contributor

History: 8 commits

ace-1's picture

Update tokenizer artifact (verified corpus retrain) + eval metrics

68abf7e verified 3 months ago

tokenizer
Fix transformers v5 auto_map + HF init 4 months ago
.gitattributes

1.52 kB
initial commit 4 months ago
README.md

1.8 kB
Publish mgpt2 tokenizer (GPT-2 exact merges) + eval metrics 4 months ago
added_tokens.json

29 Bytes
Upload mgpt2 tokenizer 4 months ago
evaluation.json

7.57 kB
Update tokenizer artifact (verified corpus retrain) + eval metrics 3 months ago
special_tokens_map.json

35 Bytes
Upload mgpt2 tokenizer 4 months ago
tokenization_mgpt2.py

80 Bytes
Publish mgpt2 tokenizer (GPT-2 exact merges) + eval metrics 4 months ago
tokenizer.model

464 kB
xet

Update tokenizer artifact (verified corpus retrain) + eval metrics 3 months ago
tokenizer.vocab

1.76 MB
Update tokenizer artifact (verified corpus retrain) + eval metrics 3 months ago
tokenizer_config.json

530 Bytes
Fix transformers v5 auto_map + HF init 4 months ago