InternLM3-8B-Instruct-Converted-Qwen2 / tokenization_internlm3.py

Commit History

use local tokenizer class
3aba99c

Zijun Xu commited on