---
library_name: transformers
datasets:
- HuggingFaceH4/MATH-500
language:
- en
base_model:
- LiquidAI/LFM2-350M-Math
---
# LFM2-350M-Math - Fine-tuned

## Model Description
This is a **LoRA fine-tuned version** of the `LiquidAI/LFM2-350M-Math` model on the **HuggingFaceH4/MATH-500** dataset.  
The model is designed to answer math questions and generate step-by-step solutions in natural language.

---

## Intended Uses
- Solve math problems directly in natural language.
- Serve as a base for further fine-tuning on other math datasets.
- Educational tools, tutoring systems, or research in automated math reasoning.

---

## Out-of-Scope Uses
- Not intended for general reasoning beyond mathematics.
- May fail or hallucinate on complex, unseen problem types.
- Should not be used for critical calculations without verification.

---

## Limitations & Biases
- The training dataset is small (500 problems), so the model may **overfit**.
- Step-by-step reasoning is learned from dataset patterns, not true reasoning.
- Accuracy can vary; always verify outputs for correctness.

---

## Training Details
- **Base Model:** LiquidAI/LFM2-350M-Math
- **Dataset:** HuggingFaceH4/MATH-500
- **Fine-tuning libraries:** transformers, datasets, peft, accelerate, bitsandbytes
- **Training arguments:**
  - num_train_epochs: 3
  - per_device_train_batch_size: 4
  - gradient_accumulation_steps: 4
  - learning_rate: 2e-4
  - fp16: True

---

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "LiquidAI/LFM2-350M-Math"
adapter_repo = "Nrex/lfm2-math-finetuned"

tokenizer = AutoTokenizer.from_pretrained(adapter_repo)
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, adapter_repo)

inputs = tokenizer("What is 12*13?", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) 
```
---
# Evaluation

- Tested on a held-out 10% split of the dataset.
- Small dataset; outputs should be verified for correctness.
- Accuracy is limited by the dataset size