Add model card with training details and graphs

README.md

@@ -1,67 +1,98 @@
 ---
 license: apache-2.0
+base_model: google/gemma-3-4b-it
 tags:
-  - lamp
-  - iot
-  - pixel-art
-  - light-control
-  - fine-tuned
-  - gguf
-  - ollama
+- gemma3
+- gguf
+- fine-tuned
+- lamp
+- lighting
+- smart-home
+- json
+datasets:
+- custom
+pipeline_tag: text-generation
 ---
 
-# LAMP Fine-Tuned Models
+# LAMP Models — Fine-tuned for Smart Lighting Control
 
-Fine-tuned language models for the **LAMP** (Moonside Lamp Emulator) project. These models generate JSON light programs from natural language descriptions.
+Fine-tuned language models that generate JSON lighting programs from natural language descriptions.
 
 ## Models
 
-| Model | Base | Parameters | GGUF Size | Final Eval Loss |
-|-------|------|-----------|-----------|-----------------|
-| `lamp-llama-3b.Q8_0.gguf` | Llama 3.2 3B Instruct | 3.2B | 3.2 GB | 0.0294 |
-| `lamp-gemma-4b.Q8_0.gguf` | Gemma 3 4B IT | 4.3B | 3.9 GB | 0.0247 |
+| Model | Base | Params | GGUF Size | Final Eval Loss |
+|-------|------|--------|-----------|-----------------|
+| **lamp-gemma-4b-v2** | Gemma 3 4B IT | 4.3B | ~4.1 GB (Q8_0) | 0.0288 |
 
 ## Training Details
 
-- **Method**: Full fine-tune (100% parameters trainable, not LoRA)
-- **Precision**: bf16
-- **Hardware**: NVIDIA H200 (140GB VRAM)
-- **Framework**: Unsloth + HuggingFace TRL
-- **Dataset**: 2,268 train / 253 validation examples
-- **Epochs**: 3
-- **Batch size**: 16 (4 per device x 4 gradient accumulation)
-- **Learning rate**: 2e-5 with cosine decay
-- **Optimizer**: AdamW
+- **Fine-tune Type:** Full parameter (no LoRA) — all 4,300,079,472 parameters trained
+- **Precision:** bf16 (bfloat16)
+- **Dataset:** 6,567 training examples + 730 validation examples
+- **Epochs:** 2
+- **Effective Batch Size:** 16 (8 per device × 2 gradient accumulation)
+- **Learning Rate:** 2e-5 with cosine schedule
+- **Optimizer:** AdamW (weight decay 0.01)
+- **Training Time:** 38.1 minutes on NVIDIA H200
+- **Peak VRAM:** 24.3 GB
 
-## Training Results
+## Training Loss
 
-![Training Results](training_results.png)
+![Training Loss](lamp-gemma-4b-v2/graphs/training_loss.png)
 
-Both models converged well:
-- **Llama 3.2 3B**: Loss 1.38 -> 0.018, eval loss 0.0294
-- **Gemma 3 4B**: Loss 1.37 -> 0.018, eval loss 0.0247
+## Training Details
 
-## Usage with Ollama
+![Training Details](lamp-gemma-4b-v2/graphs/training_details.png)
 
-1. Download the GGUF file
-2. Create a Modelfile:
+## Summary
 
-```
-# Modelfile.lamp-llama-3b
-FROM ./lamp-llama-3b.Q8_0.gguf
-PARAMETER temperature 0.3
-PARAMETER num_predict 4096
-PARAMETER stop <|eot_id|>
-```
+![Training Summary](lamp-gemma-4b-v2/graphs/training_summary.png)
+
+## Usage
+
+### With Ollama (GGUF)
 
-3. Create and run:
 ```bash
-ollama create lamp-llama-3b -f Modelfile.lamp-llama-3b
-ollama run lamp-llama-3b "warm and cozy"
+# Download the GGUF file and Modelfile from lamp-gemma-4b-v2-gguf/
+ollama create lamp-gemma -f Modelfile
+ollama run lamp-gemma "warm and cozy lighting"
+```
+
+### With Transformers (HuggingFace)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("MrMoeeee/lamp-models", subfolder="lamp-gemma-4b-v2")
+tokenizer = AutoTokenizer.from_pretrained("MrMoeeee/lamp-models", subfolder="lamp-gemma-4b-v2")
 ```
 
-## Example
+## Files
+
+```
+lamp-gemma-4b-v2/          # Full model weights + training logs
+  ├── model-00001-of-00002.safetensors
+  ├── model-00002-of-00002.safetensors
+  ├── config.json
+  ├── tokenizer.json
+  ├── training_config.json
+  ├── training_log.json
+  ├── training_metrics.csv
+  ├── metrics_detailed.json
+  └── graphs/
+      ├── training_loss.png
+      ├── training_details.png
+      └── training_summary.png
+
+lamp-gemma-4b-v2-gguf/     # Quantized GGUF for inference
+  ├── lamp-gemma-4b-v2-Q8_0.gguf
+  └── Modelfile
+```
 
-**Input**: "Create a light program for: warm and cozy"
+## Dataset
 
-**Output**: A JSON program controlling LED pixels with colors, animations, and timing for a warm ambient effect.
+The LAMP dataset consists of natural language lighting requests paired with JSON lighting programs. Each program controls RGB LEDs with support for:
+- Static colors and gradients
+- Animations (breathing, rainbow, chase, etc.)
+- Multi-step sequences with timing
+- Brightness and speed control