README.md

---
base_model: google/gemma-3-1b-it
library_name: peft
license: gemma
tags:
- lora
- 3d-printing
- microfactory
- build-small-hackathon
- peft
- chief-engineer
---

# Microfactory Node: 3D Printer (LoRA v1 — historical)

This was the first fine-tune attempt. It failed, and that failure taught me what not to do. I keep it here as a historical artifact and a reminder.

## What went wrong

I trained a LoRA on `google/gemma-3-1b-it` with rank 16 for three epochs on deterministic targets. The result parroted the same settings template for every input — it memorized, it did not judge.

## Training (for the record)

| Parameter | Value |
|-----------|-------|
| Base model | `google/gemma-3-1b-it` |
| Method | LoRA (PEFT) |
| Rank | r=16, α=32 |
| Epochs | 3 |
| Learning rate | 2e-4 |
| Dataset | Deterministic targets (single template) |
| GPU | NVIDIA A10G (24GB) |
| Framework | TRL SFTTrainer + transformers |

## Lessons learned

1. **High rank + many epochs + deterministic targets = parrot.** The model had too much capacity and too little variety. It learned one answer and repeated it.
2. **Noisy targets force judgment.** v2 switched to temperature=0.7, top_p=0.95 during dataset generation so the model cannot memorize a single template.
3. **Low rank, single epoch.** v2 used r=4 for one epoch. Less capacity, less memorization, more attention to the actual job.
4. **Base model matters.** gemma-3-1b was too small for the task. v2 moved to gemma-4-E4B-it (~4B effective).

## Do not use this adapter

Use [`microfactory-node-lora-v2`](https://huggingface.co/kylebrodeur/microfactory-node-lora-v2) or [`microfactory-node-lora-v3-qat`](https://huggingface.co/kylebrodeur/microfactory-node-lora-v3-qat) instead. This one is here for the paper trail.

## License

This adapter inherits the [Gemma license](https://ai.google.dev/gemma/terms) from its base model.