File size: 5,012 Bytes
f260793 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | ---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-230M
tags:
- lfm2
- lfm2.5
- liquid
- code
- math
- fine-tune
language:
- en
pipeline_tag: text-generation
---
# LFM2.5-230M-Code-Math
A fine-tune of [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) (the instruct-tuned model, **not** the base checkpoint) focused on strengthening code generation and math word-problem solving, while retaining the general chat and instruction-following ability of the original instruct model.
## Why this exists
LiquidAI's own model card for LFM2.5-230M states it is **not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing** — the model is tuned primarily for data extraction, structured outputs, and lightweight agentic/tool-use tasks. This fine-tune is an attempt to push a small, efficient instruct model further into code and math competence without sacrificing its existing conversational ability.
Fine-tuning started from the **instruct** checkpoint rather than the base pretrain checkpoint, specifically to preserve chat and instruction-following behavior that the base model doesn't have. An earlier fine-tune attempt starting from `LFM2.5-230M-Base` produced a model that was strong at code/math but broke down on basic conversation (e.g. echoing "Hello, who are you?" back verbatim). Starting from instruct avoided this.
## Training details
- **Base model**: `LiquidAI/LFM2.5-230M` (instruct)
- **Method**: Full fine-tune (LoRA would also work at this scale; full-FT was used here since compute wasn't a constraint)
- **Datasets**:
- Code: [`iamtarun/code_instructions_120k_alpaca`](https://huggingface.co/datasets/iamtarun/code_instructions_120k_alpaca)
- Math: [`openai/gsm8k`](https://huggingface.co/datasets/openai/gsm8k) (main split)
- **Checkpoint selection**: best checkpoint by eval loss (not final step) — training showed clear overfitting past ~step 7500, where training loss kept falling but eval loss plateaued/rose slightly. The published checkpoint is from before that point.
- **Sequence length**: 1024 tokens (dataset is short-form; base model supports up to 32K context)
- **Loss**: completion-only (loss computed only on assistant responses, not prompts)
## What it's good at
Based on manual testing across ~20+ prompts spanning algebra, geometry, general code tasks, and open-ended chat:
- **Code**: Reliable on common patterns — string/list manipulation, simple classes, recursion, file I/O, prime checking, etc. In the author's own informal side-by-side testing, output was clearer and more consistent than Qwen2.5-Coder-0.5B-Instruct on the same prompts. This is a subjective, single-user comparison, not a formal benchmark — your results may differ.
- **Math**: Grade-school word problems (gsm8k-style), percentages, basic algebra, geometry (area/perimeter) — mostly correct with gsm8k-style step annotations.
- **Chat**: Retains coherent, on-topic conversational ability inherited from the instruct base — no repetition loops or echo failures observed in testing.
- **Tool calling**: Spot-checked informally by the author using the Pythonic tool-call format LFM2.5 supports; not systematically benchmarked against other models.
## Known limitations
- Occasional arithmetic slip on multi-step algebra (e.g., correct method shown, final division not simplified).
- Not tested on data extraction or RAG.
- Still a 230M-parameter model — do not expect deep multi-step reasoning, advanced math, or long-form creative writing at the level of much larger models.
- Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "hauser458original/lfm2.5-230m-code-math"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.3, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```
GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: `hauser458original/lfm2.5-230m-code-math-GGUF`
## License
Inherits the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE) from the base model.
## Acknowledgements
Built on [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M). See the [LFM2 Technical Report](https://arxiv.org/abs/2511.23404) for details on the base architecture.
|