What this project does: Fine-tunes Liquid AI's LFM2.5-1.2B-Instruct model on Apple Silicon using HuggingFace's TRL SFTTrainer with PEFT LoRA adapters, to classify emails as spam or ham. Uses the official Liquid4All cookbook configuration.
# 1. Activate the project virtual environment
cd "spam-classifier-liquid"
source venv/bin/activate
# 2. Install dependencies (first run only)
pip install -r requirements.txt
# 3. Copy training data from the MLX project
mkdir -p training_data
cp ../spam-classifier-mlx/training_data/train.jsonl training_data/
cp ../spam-classifier-mlx/training_data/test.jsonl training_data/
# 4. Fine-tune with LoRA via TRL SFTTrainer
python3 fine_tune.py
# 5. Launch the Gradio web app
python3 app.py
# 6. Or open the notebook
jupyter notebook spam_classifier_liquid.ipynb
Model architecture note: LFM2.5 is a hybrid model that uses attention, GLU (gated linear units), and convolutional layers. That's why the LoRA target_modules list includes both q_proj/k_proj/v_proj/out_proj (attention) and w1/w2/w3 (GLU) and in_proj (conv). These exact module names come from the Liquid4All cookbook — see the guide below.
The official technical report for the LFM2 model family (the parent of LFM2.5-1.2B-Instruct). Describes the hybrid architecture — a mix of attention, convolutional, and gated linear units inspired by biological neural circuits — and the training process.
Liquid AI's official guide for fine-tuning LFM2 models with HuggingFace TRL. This is the method our project uses. Covers how to set up SFTTrainer, what LoRA parameters work well for LFM2, and example configurations.
Alternative fine-tuning method using Unsloth (2-5x faster than vanilla TRL). Not used in our project, but worth comparing if you ever want to retrain faster.
Source of our LoRA configuration. This is the official notebook that defines the exact target_modules list, rank, alpha, and training arguments the Liquid project uses. Open with Jupyter to see every step.
Alternative notebook using Unsloth for 2-5x faster training. Uses 16-bit LoRA with gradient checkpointing.
🔗 Shared References
The LoRA paper, QLoRA, the Attention Is All You Need paper, and HuggingFace PEFT/TRL/chat-template docs all apply to this project. They live in the shared folder: