---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-350M/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-350M
tags:
  - lfm2
  - lfm2.5
  - liquid
  - code
  - math
  - fine-tune
language:
  - en
pipeline_tag: text-generation
---

# LFM2.5-350M-Code-Math

A fine-tune of [LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) (instruct) focused on **multi-language code generation** and **math word-problem solving**, while retaining general chat ability through a balanced mixed dataset.

## Why this exists

An earlier 350M fine-tune (`lfm2.5-350m-python-math`) was Python-only. This version broadens code coverage to multiple languages (JavaScript, Java, C++, and others), using the same multi-language code source the [`lfm2.5-230m-code-math`](https://huggingface.co/hauser458original/lfm2.5-230m-code-math) model was trained on, while keeping the balanced-dataset approach that fixed catastrophic forgetting in earlier 350M runs (see that model's card for details on the cookie-confusion / negative-constraint failures this approach addresses).

Fine-tuning started from the **instruct checkpoint**, not base — testing confirmed base-model fine-tuning at this scale still produces alignment failures (refusals, identity confusion, math regression) even with the same mixed dataset, while the instruct checkpoint with identical data produced consistently strong, stable results.

## Training details

- **Base model:** `LiquidAI/LFM2.5-350M` (instruct)
- **Method:** Full fine-tune (96GB VRAM, no LoRA needed)
- **Datasets:**
  - Code: [`iamtarun/code_instructions_120k_alpaca`](https://huggingface.co/datasets/iamtarun/code_instructions_120k_alpaca) (multi-language: Python, JavaScript, Java, C++, and others; 60k sampled)
  - Math: [`openai/gsm8k`](https://huggingface.co/datasets/openai/gsm8k) (main split)
  - General Chat: [`yahma/alpaca-cleaned`](https://huggingface.co/datasets/yahma/alpaca-cleaned) (30k sample subset, prevents knowledge/chat forgetting)
  - Custom Fix-It: Hand-crafted examples for negative constraints ("no dairy", "no eggs") and complete runnable Pygame scripts (duplicated 50x for weight)
- **Checkpoint selection:** Best by eval_loss
- **Sequence length:** 2048 tokens (accommodates full scripts without truncation)
- **Max response chars:** 3500
- **Epochs:** 2 (higher epoch counts showed overfitting in earlier same-family runs)
- **Learning rate:** 2e-5
- **Loss:** Completion-only

## What it's good at

Based on manual testing across chat, code, and math prompts:

- **Code**: Correct, clean output across common patterns — string/list manipulation, classes, file I/O, recursion, simple game loops (Pygame). Broader language coverage than the Python-only predecessor, matching the multi-language dataset used by the 230M model.
- **Math**: Full correctness on grade-school word problems, percentages, and multi-step algebra (including a rectangle-area and multi-step algebra pattern that tripped up earlier smaller checkpoints) — all resolved cleanly here, with proper gsm8k-style step annotations.
- **Chat**: Coherent, mature responses. Notably handles uncertainty well — e.g., when asked to match sock colors to an unspecified shirt color, it correctly points out it doesn't know the shirt's actual color rather than hallucinating one, and offers a general framework instead.

## Known limitations

- Non-Python language coverage (JS, Java, C++, etc.) has not been as extensively manually spot-checked as Python; results may vary more by language.
- Occasional minor scope-narrowing on ambiguous prompts (e.g., a "simple calculator" request returned a single-operation function rather than a full four-operation calculator) — functional but not always fully matching the implied scope of a request.
- Still a 350M-parameter model — do not expect deep multi-step reasoning or long-form creative writing at the level of much larger models.
- Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards.
- Not tested on data extraction, RAG, or tool-calling formats.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hauser458original/lfm2.5-350m-code-math"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [{"role": "user", "content": "Write a JavaScript function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

output = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.5, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: [`hauser458original/lfm2.5-350m-code-math-GGUF`](https://huggingface.co/hauser458original/lfm2.5-350m-code-math-GGUF)

## License

Inherits the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-350M/blob/main/LICENSE) from the base model.

## Acknowledgements

Built on [LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M). See the [LFM2 Technical Report](https://arxiv.org/abs/2511.23404) for details on the base architecture.