---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-230M
tags:
  - lfm2
  - lfm2.5
  - liquid
  - code
  - math
  - fine-tune
language:
  - en
pipeline_tag: text-generation
---

# LFM2.5-230M-Code-Math

A fine-tune of [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) (the instruct-tuned model, **not** the base checkpoint) focused on strengthening code generation and math word-problem solving, while retaining the general chat and instruction-following ability of the original instruct model.

## Why this exists

LiquidAI's own model card for LFM2.5-230M states it is **not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing** — the model is tuned primarily for data extraction, structured outputs, and lightweight agentic/tool-use tasks. This fine-tune is an attempt to push a small, efficient instruct model further into code and math competence without sacrificing its existing conversational ability.

Fine-tuning started from the **instruct** checkpoint rather than the base pretrain checkpoint, specifically to preserve chat and instruction-following behavior that the base model doesn't have. An earlier fine-tune attempt starting from `LFM2.5-230M-Base` produced a model that was strong at code/math but broke down on basic conversation (e.g. echoing "Hello, who are you?" back verbatim). Starting from instruct avoided this.

## Training details

- **Base model**: `LiquidAI/LFM2.5-230M` (instruct)
- **Method**: Full fine-tune (LoRA would also work at this scale; full-FT was used here since compute wasn't a constraint)
- **Datasets**:
  - Code: [`iamtarun/code_instructions_120k_alpaca`](https://huggingface.co/datasets/iamtarun/code_instructions_120k_alpaca)
  - Math: [`openai/gsm8k`](https://huggingface.co/datasets/openai/gsm8k) (main split)
- **Checkpoint selection**: best checkpoint by eval loss (not final step) — training showed clear overfitting past ~step 7500, where training loss kept falling but eval loss plateaued/rose slightly. The published checkpoint is from before that point.
- **Sequence length**: 1024 tokens (dataset is short-form; base model supports up to 32K context)
- **Loss**: completion-only (loss computed only on assistant responses, not prompts)

## What it's good at

Based on manual testing across ~20+ prompts spanning algebra, geometry, general code tasks, and open-ended chat:

- **Code**: Reliable on common patterns — string/list manipulation, simple classes, recursion, file I/O, prime checking, etc. In the author's own informal side-by-side testing, output was clearer and more consistent than Qwen2.5-Coder-0.5B-Instruct on the same prompts. This is a subjective, single-user comparison, not a formal benchmark — your results may differ.
- **Math**: Grade-school word problems (gsm8k-style), percentages, basic algebra, geometry (area/perimeter) — mostly correct with gsm8k-style step annotations.
- **Chat**: Retains coherent, on-topic conversational ability inherited from the instruct base — no repetition loops or echo failures observed in testing.
- **Tool calling**: Spot-checked informally by the author using the Pythonic tool-call format LFM2.5 supports; not systematically benchmarked against other models.

## Known limitations

- Occasional arithmetic slip on multi-step algebra (e.g., correct method shown, final division not simplified).
- Not tested on data extraction or RAG.
- Still a 230M-parameter model — do not expect deep multi-step reasoning, advanced math, or long-form creative writing at the level of much larger models.
- Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hauser458original/lfm2.5-230m-code-math"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.3, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: `hauser458original/lfm2.5-230m-code-math-GGUF`

## License

Inherits the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE) from the base model.

## Acknowledgements

Built on [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M). See the [LFM2 Technical Report](https://arxiv.org/abs/2511.23404) for details on the base architecture.