nanochat-d34-rl-hf

This is a pankajmathur/nanochat-d34-rl converted to HuggingFace transformers format.

Usage

Install Transformer Library from Github with nanochat support

!pip install -q git+https://github.com/huggingface/transformers.git

Use dedicated NanoChatForCausalLM and PreTrainedTokenizerFast packages from Transformer Library

import torch
from transformers import NanoChatForCausalLM, PreTrainedTokenizerFast

# Load the converted model and tokenizer
tokenizer = PreTrainedTokenizerFast.from_pretrained("pankajmathur/nanochat-d34-rl-hf")
model = NanoChatForCausalLM.from_pretrained(
    "pankajmathur/nanochat-d34-rl-hf",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Generate text
prompt = """
Julie is reading a 120-page book. Yesterday, she was able to read 12 pages and today,
she read twice as many pages as yesterday.
If she wants to read half of the remaining pages tomorrow, how many pages should she read?
"""

inputs = tokenizer(prompt, return_tensors="pt")
input_ids = inputs["input_ids"].to(model.device)

with torch.no_grad():
    outputs = model.generate(
        input_ids,
        max_new_tokens=8192,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"🤖 Response:\n{response}")

License

MIT License