harshbhatt7585/arithmetic-king-1b

Arithmetic King 1B

Repository id: harshbhatt7585/arithmetic-king-1b.

This model is a PEFT LoRA adapter trained with TRL GRPO on synthetic arithmetic episodes. It is tuned to answer in XML format:

• <reasoning>...</reasoning>
• <answer>...</answer>

Artifact Type

This repo contains adapter weights only (not full base model weights). Use with base model:

• meta-llama/Llama-3.2-1B-Instruct

Training Configuration

• Trainer: TRL GRPOTrainer
• Fine-tuning method: LoRA (PEFT)
• Environment: arithmetic reasoning episodes
• Reward: correctness reward + XML-format bonus
• Output style target: short reasoning plus final integer answer

Intended Use

• Arithmetic-reasoning RLVR experiments
• GRPO/LoRA workflow demonstrations
• Adapter-centric fine-tuning studies on small instruct models

Limitations

• Trained on synthetic arithmetic prompts only
• Limited transfer to broader reasoning/math tasks
• May produce malformed XML or incorrect answers
• Not suitable for high-stakes use

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "meta-llama/Llama-3.2-1B-Instruct"
adapter_id = "harshbhatt7585/arithmetic-king-1b"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Solve: (12 + 3) * 2. Return XML with <reasoning> and <answer>."
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=True))

License

Adapter usage inherits base model license and terms:

• meta-llama/Llama-3.2-1B-Instruct