RRT-Foundation / tokenizer_pointer.txt

Open release: RRT-355M weights, CORE eval artifacts, README

4f983a9 verified 11 days ago

151 Bytes

	Tokenizer: openai-community/gpt2 (canonical GPT-2 BPE, vocab 50257).
	Load via `transformers.AutoTokenizer.from_pretrained('openai-community/gpt2')`.