Hugging Face's logo

RWKV
/
v6-Finch-3B-HF

Text Generation
Transformers
PyTorch
rwkv6
custom_code
Model card Files
xet
Community
4

Update hf_rwkv_tokenizer.py

#3
by KoichiYasuoka - opened
base: refs/heads/main
←
from: refs/pr/3
Discussion Files changed
+8
-1
KoichiYasuoka

Bug fix of convert_tokens_to_ids and decode for byte-fallback characters.

 Update hf_rwkv_tokenizer.py089d741a
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment