File size: 1,901 Bytes
4ee0e8f 3540c38 4ee0e8f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
base_model: meta-llama/Llama-3.2-1B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
- grpo
- lora
- trl
- arithmetic
- reasoning
language:
- en
---
# Arithmetic King 1B
Repository id: `harshbhatt7585/arithmetic-king-1b`.
This model is a PEFT LoRA adapter trained with TRL GRPO on synthetic arithmetic episodes. It is tuned to answer in XML format:
- `<reasoning>...</reasoning>`
- `<answer>...</answer>`
## Artifact Type
This repo contains **adapter weights only** (not full base model weights). Use with base model:
- `meta-llama/Llama-3.2-1B-Instruct`
## Training Configuration
- Trainer: TRL `GRPOTrainer`
- Fine-tuning method: LoRA (PEFT)
- Environment: arithmetic reasoning episodes
- Reward: correctness reward + XML-format bonus
- Output style target: short reasoning plus final integer answer
## Intended Use
- Arithmetic-reasoning RLVR experiments
- GRPO/LoRA workflow demonstrations
- Adapter-centric fine-tuning studies on small instruct models
## Limitations
- Trained on synthetic arithmetic prompts only
- Limited transfer to broader reasoning/math tasks
- May produce malformed XML or incorrect answers
- Not suitable for high-stakes use
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
base_id = "meta-llama/Llama-3.2-1B-Instruct"
adapter_id = "harshbhatt7585/arithmetic-king-1b"
tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, adapter_id)
prompt = "Solve: (12 + 3) * 2. Return XML with <reasoning> and <answer>."
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```
## License
Adapter usage inherits base model license and terms:
- `meta-llama/Llama-3.2-1B-Instruct`
|