| --- |
| license: other |
| license_name: lfm1.0 |
| license_link: https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE |
| base_model: LiquidAI/LFM2.5-230M |
| tags: |
| - lfm2 |
| - lfm2.5 |
| - liquid |
| - code |
| - math |
| - fine-tune |
| language: |
| - en |
| pipeline_tag: text-generation |
| --- |
| |
| # LFM2.5-230M-Code-Math |
|
|
| A fine-tune of [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) (the instruct-tuned model, **not** the base checkpoint) focused on strengthening code generation and math word-problem solving, while retaining the general chat and instruction-following ability of the original instruct model. |
|
|
| ## Why this exists |
|
|
| LiquidAI's own model card for LFM2.5-230M states it is **not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing** — the model is tuned primarily for data extraction, structured outputs, and lightweight agentic/tool-use tasks. This fine-tune is an attempt to push a small, efficient instruct model further into code and math competence without sacrificing its existing conversational ability. |
|
|
| Fine-tuning started from the **instruct** checkpoint rather than the base pretrain checkpoint, specifically to preserve chat and instruction-following behavior that the base model doesn't have. An earlier fine-tune attempt starting from `LFM2.5-230M-Base` produced a model that was strong at code/math but broke down on basic conversation (e.g. echoing "Hello, who are you?" back verbatim). Starting from instruct avoided this. |
|
|
| ## Training details |
|
|
| - **Base model**: `LiquidAI/LFM2.5-230M` (instruct) |
| - **Method**: Full fine-tune (LoRA would also work at this scale; full-FT was used here since compute wasn't a constraint) |
| - **Datasets**: |
| - Code: [`iamtarun/code_instructions_120k_alpaca`](https://huggingface.co/datasets/iamtarun/code_instructions_120k_alpaca) |
| - Math: [`openai/gsm8k`](https://huggingface.co/datasets/openai/gsm8k) (main split) |
| - **Checkpoint selection**: best checkpoint by eval loss (not final step) — training showed clear overfitting past ~step 7500, where training loss kept falling but eval loss plateaued/rose slightly. The published checkpoint is from before that point. |
| - **Sequence length**: 1024 tokens (dataset is short-form; base model supports up to 32K context) |
| - **Loss**: completion-only (loss computed only on assistant responses, not prompts) |
|
|
| ## What it's good at |
|
|
| Based on manual testing across ~20+ prompts spanning algebra, geometry, general code tasks, and open-ended chat: |
|
|
| - **Code**: Reliable on common patterns — string/list manipulation, simple classes, recursion, file I/O, prime checking, etc. In the author's own informal side-by-side testing, output was clearer and more consistent than Qwen2.5-Coder-0.5B-Instruct on the same prompts. This is a subjective, single-user comparison, not a formal benchmark — your results may differ. |
| - **Math**: Grade-school word problems (gsm8k-style), percentages, basic algebra, geometry (area/perimeter) — mostly correct with gsm8k-style step annotations. |
| - **Chat**: Retains coherent, on-topic conversational ability inherited from the instruct base — no repetition loops or echo failures observed in testing. |
| - **Tool calling**: Spot-checked informally by the author using the Pythonic tool-call format LFM2.5 supports; not systematically benchmarked against other models. |
|
|
| ## Known limitations |
|
|
| - Occasional arithmetic slip on multi-step algebra (e.g., correct method shown, final division not simplified). |
| - Not tested on data extraction or RAG. |
| - Still a 230M-parameter model — do not expect deep multi-step reasoning, advanced math, or long-form creative writing at the level of much larger models. |
| - Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "hauser458original/lfm2.5-230m-code-math" |
| model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto") |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| |
| messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}] |
| inputs = tokenizer.apply_chat_template( |
| messages, add_generation_prompt=True, return_tensors="pt", return_dict=True |
| ).to(model.device) |
| |
| output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.3, top_p=0.9) |
| print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)) |
| ``` |
|
|
| GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: `hauser458original/lfm2.5-230m-code-math-GGUF` |
| |
| ## License |
| |
| Inherits the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE) from the base model. |
| |
| ## Acknowledgements |
| |
| Built on [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M). See the [LFM2 Technical Report](https://arxiv.org/abs/2511.23404) for details on the base architecture. |
| |
