---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-350M/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-350M
tags:
  - lfm2
  - lfm2.5
  - liquid
  - python
  - math
  - fine-tune
language:
  - en
pipeline_tag: text-generation
---

# LFM2.5-350M-Python-Math

A fine-tune of [LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) (instruct) focused on **Python code generation** and **math word-problem solving**, while retaining general chat ability through a balanced mixed dataset.

## Why this exists

The previous 230M fine-tune (`lfm2.5-230m-code-math`) showed strong potential but suffered from catastrophic forgetting (e.g., confusing baking cookies with HTTP cookies, failing negative constraints like "no dairy"). This 350M version addresses those issues by:

1. **Mixing general chat data** (`yahma/alpaca-cleaned`, 30k samples) to prevent knowledge loss.
2. **Injecting custom fix-it examples** targeting specific failure modes (negative constraints, complete Pygame scripts).
3. **Using longer context** (2048 tokens) so code outputs aren't truncated mid-function.
4. **Reducing epochs to 2** with a lower learning rate (`2e-5`) to prevent overfitting observed in earlier runs.

Fine-tuning started from the **instruct checkpoint** rather than base. Testing confirmed that at 350M scale, starting from base with a mixed dataset still produced alignment failures (refusals, identity confusion, math regression), while the instruct checkpoint with the same dataset produced consistently strong results.

## Training details

- **Base model:** `LiquidAI/LFM2.5-350M` (instruct)
- **Method:** Full fine-tune (96GB VRAM, no LoRA needed)
- **Datasets:**
  - Python Code: [`iamtarun/python_code_instructions_18k_alpaca`](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca) (Python-focused, replacing the multi-language 120k set)
  - Math: [`openai/gsm8k`](https://huggingface.co/datasets/openai/gsm8k) (main split)
  - General Chat: [`yahma/alpaca-cleaned`](https://huggingface.co/datasets/yahma/alpaca-cleaned) (30k sample subset)
  - Custom Fix-It: Hand-crafted examples for negative constraints ("no dairy", "no eggs") and complete runnable Pygame scripts (duplicated 50x for weight)
- **Checkpoint selection:** Best by eval_loss
- **Sequence length:** 2048 tokens (increased from 1024 to accommodate full scripts)
- **Max response chars:** 3500 (prevents code truncation)
- **Epochs:** 2 (reduced from 4; overfitting observed past epoch 2 in prior runs)
- **Learning rate:** 2e-5 (reduced from 5e-5 for 350M stability)
- **Loss:** Completion-only

## What it's good at

- **Python Code**: Complete, runnable scripts including Pygame game loops, file I/O, classes, list comprehensions, and algorithmic implementations (e.g., two-pointer palindrome check). No more placeholder `pass` statements or truncated functions.
- **Math**: GSM8K-style word problems with step-by-step reasoning annotations (`<<...>>`). Reliable on algebra, percentages, geometry, and multi-step arithmetic.
- **General Chat**: Retains coherent conversational ability. Correctly handles negative constraints (e.g., "breakfast without eggs" returns egg-free options). Knows the difference between baking cookies and browser cookies.
- **Speed**: At 350M parameters, achieves ~157 t/s generation on laptop CPU (i5-12450H) with Q5_K_S quantization via llama.cpp.

## Known limitations

- **Python only**: Trained exclusively on Python code instructions. Other languages were not included in this fine-tune.
- **Sentence counting**: May not strictly adhere to "exactly N sentences" constraints.
- **Identity**: May occasionally claim to be developed by Google (artifact from Alpaca-Cleaned training data).
- **Still 350M parameters**: Do not expect deep multi-step reasoning or long-form creative writing at the level of larger models.
- Not evaluated on safety-critical, medical, or legal use cases.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hauser458original/lfm2.5-350m-python-math"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

output = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.5, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: [`hauser458original/lfm2.5-350m-python-math-GGUF`](https://huggingface.co/hauser458original/lfm2.5-350m-python-math-GGUF)

## License

Inherits the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-350M/blob/main/LICENSE) from the base model.

## Acknowledgements

Built on [LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M). See the [LFM2 Technical Report](https://arxiv.org/abs/2511.23404) for details on the base architecture.