Upload folder using huggingface_hub

f260793 verified about 16 hours ago

5.01 kB

license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-230M
tags:
  - lfm2
  - lfm2.5
  - liquid
  - code
  - math
  - fine-tune
language:
  - en
pipeline_tag: text-generation

LFM2.5-230M-Code-Math

A fine-tune of LiquidAI/LFM2.5-230M (the instruct-tuned model, not the base checkpoint) focused on strengthening code generation and math word-problem solving, while retaining the general chat and instruction-following ability of the original instruct model.

Why this exists

LiquidAI's own model card for LFM2.5-230M states it is not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing — the model is tuned primarily for data extraction, structured outputs, and lightweight agentic/tool-use tasks. This fine-tune is an attempt to push a small, efficient instruct model further into code and math competence without sacrificing its existing conversational ability.

Fine-tuning started from the instruct checkpoint rather than the base pretrain checkpoint, specifically to preserve chat and instruction-following behavior that the base model doesn't have. An earlier fine-tune attempt starting from LFM2.5-230M-Base produced a model that was strong at code/math but broke down on basic conversation (e.g. echoing "Hello, who are you?" back verbatim). Starting from instruct avoided this.

Training details

Base model: LiquidAI/LFM2.5-230M (instruct)
Method: Full fine-tune (LoRA would also work at this scale; full-FT was used here since compute wasn't a constraint)
Datasets:
- Code: iamtarun/code_instructions_120k_alpaca
- Math: openai/gsm8k (main split)
Checkpoint selection: best checkpoint by eval loss (not final step) — training showed clear overfitting past ~step 7500, where training loss kept falling but eval loss plateaued/rose slightly. The published checkpoint is from before that point.
Sequence length: 1024 tokens (dataset is short-form; base model supports up to 32K context)
Loss: completion-only (loss computed only on assistant responses, not prompts)

What it's good at

Based on manual testing across ~20+ prompts spanning algebra, geometry, general code tasks, and open-ended chat:

Code: Reliable on common patterns — string/list manipulation, simple classes, recursion, file I/O, prime checking, etc. In the author's own informal side-by-side testing, output was clearer and more consistent than Qwen2.5-Coder-0.5B-Instruct on the same prompts. This is a subjective, single-user comparison, not a formal benchmark — your results may differ.
Math: Grade-school word problems (gsm8k-style), percentages, basic algebra, geometry (area/perimeter) — mostly correct with gsm8k-style step annotations.
Chat: Retains coherent, on-topic conversational ability inherited from the instruct base — no repetition loops or echo failures observed in testing.
Tool calling: Spot-checked informally by the author using the Pythonic tool-call format LFM2.5 supports; not systematically benchmarked against other models.

Known limitations

Occasional arithmetic slip on multi-step algebra (e.g., correct method shown, final division not simplified).
Not tested on data extraction or RAG.
Still a 230M-parameter model — do not expect deep multi-step reasoning, advanced math, or long-form creative writing at the level of much larger models.
Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hauser458original/lfm2.5-230m-code-math"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.3, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: hauser458original/lfm2.5-230m-code-math-GGUF

License

Inherits the LFM Open License v1.0 from the base model.

Acknowledgements

Built on LiquidAI/LFM2.5-230M. See the LFM2 Technical Report for details on the base architecture.