penfever's picture
Add GLM 5.2 evaluation tokenizer metadata
5fed523 verified
|
Raw
History Blame Contribute Delete
600 Bytes
---
license: mit
base_model: cyankiwi/GLM-5.2-AWQ-INT4
---
# GLM 5.2 evaluation tokenizer
Tokenizer-only artifact for evaluating `cyankiwi/GLM-5.2-AWQ-INT4` with
Transformers 4.57.6. `tokenizer.json` and `chat_template.jinja` are copied
unchanged from model revision `956f4d228e36c95f66a80058fe64b69af67360ca`.
The source metadata names the unavailable `TokenizersBackend` class and stores
`extra_special_tokens` as a list, which Transformers 4.57.6 rejects. This
artifact names `PreTrainedTokenizerFast` and removes that redundant field. The
vocabulary and rendered chat templates are unchanged.