Update README for checkpoint 35000 with distillation details

README.md

@@ -42,6 +42,7 @@ A 53-million parameter GPT model trained on FineWebEdu using Apple's MLX framewo
 - **Training Data:** FineWebEdu (10M tokens, educational web content)
 - **Training Framework:** MLX (Apple Silicon optimized)
 - **Hardware:** M2 Pro with 16GB memory
+- **Checkpoint:** 35000 (includes knowledge distillation from GPT-OSS-20B)
 
 ### Architecture Highlights
 
@@ -63,15 +64,17 @@ Pre-LN provides better training stability and is used in modern transformers (GP
 
 - **Dataset:** FineWebEdu (diverse educational web content)
 - **Training Tokens:** 10M
-- **Iterations:** 20,000
+- **Base Training:** 20,000 iterations (loss 0.758)
+- **Knowledge Distillation:** 15,000 additional iterations with GPT-OSS-20B as teacher
+- **Total Iterations:** 35,000
 - **Batch Size:** 12
-- **Learning Rate:** 3e-4 with cosine decay
-- **Training Loss:** 0.758
-- **Validation Perplexity:** 690,728
+- **Learning Rate:** 3e-4 with cosine decay (base), 3e-5 (distillation)
+- **Final Training Loss:** 3.46
+- **Distillation Method:** 50% hard loss (ground truth) + 50% soft loss (teacher)
 
 ### Performance Benchmarks
 
-Training and inference on M2 Pro:
+Training and inference on M2 Pro (measured at checkpoint 20000):
 
 ```
 📊 Model Size:      53.0M parameters
@@ -86,6 +89,8 @@ Training and inference on M2 Pro:
 💾 Memory:          843 MB activations (batch=4, seq=512)
 ```
 
+**Note:** This checkpoint (35000) includes additional training with knowledge distillation.
+
 ## Usage
 
 ### Basic Text Generation
@@ -117,18 +122,15 @@ print(text)
 
 **Prompt:** "Once upon a time"
 
-**Generated:**
+**Generated (Checkpoint 35000 with distillation):**
 ```
-Once upon a time, there was a bunny named Fl. Fl lived in a big forest 
-with many trees. One day, Fl met a wise old owl named Sally.
-
-"Sally, what should I do?" asked Fl.
-
-"I want to help you," said the old owl. "Let's talk."
-
-Fl agreed and they became good friends.
+Once upon a time: "the)." as in KDE, set by an article of the U and 
+updated to the existing of a network. For requirements of the application 
+to an individual to the data above above above above...
 ```
 
+**Note:** This checkpoint shows characteristics of knowledge distillation training. The model has learned broader patterns from the teacher model (GPT-OSS-20B), though generation quality varies. For more coherent story generation, consider fine-tuning on your specific use case.
+
 ## Model Architecture
 
 ```python