File size: 1,797 Bytes
aa71027 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
base_model: Qwen/Qwen2.5-7B-Instruct
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
tags:
- lora
- sft
- transformers
- trl
- unsloth
- bloom-taxonomy
- sna-learning
datasets:
- Dev-the-dev91/sna-regal-training-data
---
# SNA Learning — Bloom's Taxonomy Stage 2 (Apply + Analyze)
LoRA adapter for personalized CS/ML concept teaching using Bloom's Taxonomy scaffolding and Netflix-anchored memory palaces.
## Training Details
- **Base model:** Qwen/Qwen2.5-7B-Instruct (4-bit via Unsloth)
- **Stage:** 2 of 3 (Apply + Analyze levels), continuing from Stage 1 (Remember + Understand)
- **Method:** SFT with LoRA (rank 32, alpha 32, dropout 0.05)
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Epochs:** 3
- **Learning rate:** 1e-4 (cosine schedule, 4 warmup steps)
- **Batch size:** 1 × 8 gradient accumulation
- **Max sequence length:** 1024
- **Precision:** bf16
- **Hardware:** Modal (GPU)
- **Training time:** ~789s (~13 min)
- **Framework:** TRL + PEFT + Unsloth
## Metrics
| Metric | Value |
|--------|-------|
| Train loss (avg) | 0.5397 |
| Train loss (final step) | 0.4071 |
| Eval loss | 0.7310 |
| Grad norm (final) | 0.336 |
| Total steps | 165 |
## Usage
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "Dev-the-dev91/sna-bloom-stage2")
tokenizer = AutoTokenizer.from_pretrained("Dev-the-dev91/sna-bloom-stage2")
```
## Bloom's Levels Covered
- **Stage 1:** Remember + Understand (recall, explain, mnemonic, song)
- **Stage 2 (this):** Apply + Analyze (scenario walkthrough, component decomposition)
- **Stage 3:** Evaluate + Create (planned)
|