---
library_name: transformers
pipeline_tag: text-generation
base_model: LiquidAI/LFM2.5-350M
tags:
- causal-lm
- sft
- math
- chatml
- transformers
---

# Math Curated SFT

This is a full-model SFT checkpoint trained from `LiquidAI/LFM2.5-350M` on
`User01110/math-curated-dataset`.

## Training

- Method: TRL `SFTTrainer`
- Dataset split: `train`
- Training rows: 39040
- Epochs: 1
- Max sequence length: 1024
- Target style: full generated response
- Format: the base tokenizer chat template via `tokenizer.apply_chat_template`
- System prompt: `You are a math-focused assistant. Solve the user's math problem and follow the training format: Understanding Query, Drafting Answer, Refining The Answer, and Final Response.`

## Format

Each row is formatted with:

```python
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": prompt},
]
prompt_text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
training_text = prompt_text + response + (tokenizer.eos_token or "")
```

## Important limitation

This model is trained on generated math-style data. Responses may contain
incorrect arithmetic or flawed reasoning, and should not be treated as reliable
mathematical answers without independent verification.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "User01110/LFM-2.5-350M-MathMini"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

messages = [
    {"role": "system", "content": "You are a math-focused assistant. Solve the user's math problem and follow the training format: Understanding Query, Drafting Answer, Refining The Answer, and Final Response."},
    {"role": "user", "content": "John has 22 apples, he eats 10 of them, how many apples does john have now?"},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.1,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
```