File size: 5,409 Bytes
38b4eff | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | # NeuralAI — Training Metrics & Documentation
**Last Updated: May 18, 2026**
---
## 📈 Training Results
| Metric | Value |
| --- | --- |
| **Final Loss** | 0.040 |
| **Loss Reduction** | 98% from baseline |
| **Training Samples** | 347 (SFT) + 159 (DPO) |
| **Validation Split** | 20% |
| **Training Time** | \~45 min (Colab T4 GPU) |
| **Perplexity** | Tracked per epoch |
| **Hardware** | NVIDIA T4 (Google Colab) |
---
## 🧠 Dataset Breakdown (347 samples)
| Category | Count | Topics |
| --- | --- | --- |
| **Coding** | 80 | Python, JavaScript, SQL, REST APIs, debugging, code review |
| **ML/AI** | 45 | Transformers, RAG, fine-tuning, NLP, neural networks |
| **Data Science** | 40 | Pandas, NumPy, visualization, statistics, data cleaning |
| **Web Dev** | 35 | HTML/CSS, React, Flask, APIs, deployment |
| **General Q&A** | 50 | Concepts, explanations, comparisons, how-it-works |
| **Writing** | 35 | Emails, essays, reports, documentation |
| **System Admin** | 25 | Linux, Docker, networking, troubleshooting |
| **Math/Logic** | 37 | Algorithms, data structures, calculus, proofs |
---
## 📈 DPO Alignment Results (v5.0)
| Metric | Value | Notes |
| --- | --- | --- |
| **DPO Phase 1** | COMPLETE | 159 preference pairs |
| **Loss (DPO)** | 0.69 → 0.29 | Significant convergence |
| **Margin** | +1.10 | Model clearly distinguishes "chosen" vs "rejected" |
| **Accuracy** | 100% | Final batch accuracy on training set |
| **Categories** | 8 | Code, Logic, Reasoning, etc. |
---
## 🏗️ Proposed DPO Categories for Expansion
To improve model alignment and personality, the following categories are proposed for the next DPO phase:
1. **Memphis Culture** - Knowledge about Memphis, TN, history, music (blues/soul), and the Founder's background.
2. **AI Ethics & Safety** - Refusing harmful requests, maintaining helpful yet bounded behavior.
3. **Code Optimization** - Preferring performant, idiomatic code over naive implementations.
4. **System Architecture** - Designing scalable systems and proper abstractions.
5. **Multi-step Reasoning** - Better handling of complex, multi-part instructions.
6. **Creative Writing & Roleplay** - More engaging and personality-driven responses.
7. **Advanced Debugging** - Identifying subtle bugs and suggesting fixes with rationale.
---
## 🏗️ Training Configuration
```python
# Model
base_model = "HuggingFaceTB/SmolLM2-360M-Instruct"
# Quantization (QLoRA)
load_in_4bit = True
bnb_4bit_quant_type = "nf4"
bnb_4bit_compute_dtype = "float16"
# LoRA
lora_rank = 16
lora_alpha = 32
lora_dropout = 0.05
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
# Training
learning_rate = 2e-4
lr_scheduler = "cosine" # with warmup
warmup_ratio = 0.1
per_device_train_batch_size = 4
gradient_accumulation_steps = 4
num_train_epochs = 3
max_grad_norm = 1.0 # gradient clipping
fp16 = True # mixed precision
# Data
max_length = 512
train_validation_split = 0.2
```
---
## 📉 Loss Progression
| Epoch | Train Loss | Val Loss | Notes |
| --- | --- | --- | --- |
| 1 | 2.10 | 0.85 | Warmup complete |
| 2 | 0.65 | 0.18 | Rapid learning phase |
| 3 | 0.12 | 0.04 | Convergence |
**Final:** Loss 0.040 — model converges well with no overfitting (val loss close to train loss).
---
## 🔧 Training Pipeline
```markdown
1. Data Preparation
└── train.jsonl (347 JSON samples, ChatML format)
2. Environment Setup
└── pip install torch transformers peft bitsandbytes accelerate datasets
3. Script Execution
└── python train_neuralai.py --epochs 3 --batch-size 4 --lr 2e-4
4. Model Output
└── checkpoints/final_model/
├── adapter_model.safetensors (LoRA weights)
├── adapter_config.json (LoRA config)
├── tokenizer.json
└── tokenizer_config.json
5. Deployment
└── Upload adapter to HuggingFace Hub
└── Deploy via Flask web UI
```
---
## ✅ Features Implemented
- ✅ LR scheduler with warmup
- ✅ 20% validation split
- ✅ Perplexity metrics (logged per epoch)
- ✅ Gradient clipping (max_norm=1.0)
- ✅ InstructionDataset class (ChatML format)
- ✅ QLoRA fine-tuning (4-bit NF4)
- ✅ Float16 training (no bitsandbytes CUDA issues)
- ✅ Flash Attention / SDPA fallback
---
## 🚨 Colab Issues Fixed (For Reference)
| Error | Fix |
| --- | --- |
| `output.input_ids[..., -1]` shape mismatch | Used `attn_implementation="eager"` |
| SDPA `torch.compile` compatibility | Added `torch.compile` fallback |
| Unused column removal crash | Set `remove_unused_columns=False` |
| `bitsandbytes` CUDA errors | Switched to float16 (no quantization) |
| Template mismatch on generation | Used `apply_chat_template` with try/except |
---
## 📦 Dependencies
```markdown
torch>=2.0
transformers>=4.40
peft>=0.10
datasets>=2.18
chromadb>=0.4
sentence-transformers>=2.3
pypdf>=4.0
python-docx>=1.0
flask>=3.0
```
---
## 🎯 Next Steps
1. **DPO Expansion** — Add more preference pairs in the proposed categories (Target: 500+ pairs).
2. **Streaming Optimization** — Refine the streaming UI for faster perceived latency.
3. **Tool Use Alignment** — Train specifically on `<tool>` tag usage and results.
4. **Automated Evaluation** — Implement a "Model vs Model" evaluation pipeline.
5. **GPU Serving** — Migrate to a persistent GPU-enabled environment. |