End of training

README.md

@@ -37,6 +37,7 @@ test_datasets:
   - path: DannyAI/African-History-QA-Dataset
     split: validation
     type: alpaca_chat.load_qa
+    # Fixed the missing quote and indentation below
     system_prompt: "You are a helpful AI assistant specialised in African history which gives concise answers to questions asked"
 
 # 2. Output & Chat Configuration
@@ -64,8 +65,10 @@ bf16: true
 fp16: false
 
 # 6. Training Duration & Optimizer
-num_epochs: 10
-warmup_steps: 10
+max_steps: 650  
+# removed
+# num_epochs: 
+warmup_steps: 20
 learning_rate: 0.00002
 optimizer: adamw_torch 
 lr_scheduler: cosine
@@ -73,10 +76,11 @@ lr_scheduler: cosine
 # 7. Logging & Evaluation
 wandb_project: phi4_african_history
 wandb_name: phi4_lora_axolotl
+
 eval_strategy: steps
-eval_steps: 100
+eval_steps: 50
 save_strategy: steps
-save_steps: 200
+save_steps: 100
 logging_steps: 5
 
 # 8. Public Hugging Face Hub Upload
@@ -92,11 +96,11 @@ hub_private_repo: false
 
 This model is a fine-tuned version of [microsoft/Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) on the DannyAI/African-History-QA-Dataset dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.0938
-- Ppl: 8.1156
+- Loss: 1.7479
+- Ppl: 5.7428
 - Memory/max Active (gib): 14.84
 - Memory/max Allocated (gib): 14.84
-- Memory/device Reserved (gib): 31.82
+- Memory/device Reserved (gib): 31.79
 
 ## Model description
 
@@ -123,15 +127,27 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 8
 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 10
-- training_steps: 120
+- lr_scheduler_warmup_steps: 20
+- training_steps: 650
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss | Ppl    | Active (gib) | Allocated (gib) | Reserved (gib) |
-|:-------------:|:------:|:----:|:---------------:|:------:|:------------:|:---------------:|:--------------:|
-| No log        | 0      | 0    | 2.1335          | 8.4442 | 14.82        | 14.82           | 15.37          |
-| 4.9679        | 7.7059 | 100  | 2.0938          | 8.1156 | 14.84        | 14.84           | 31.82          |
+| Training Loss | Epoch   | Step | Validation Loss | Ppl    | Active (gib) | Allocated (gib) | Reserved (gib) |
+|:-------------:|:-------:|:----:|:---------------:|:------:|:------------:|:---------------:|:--------------:|
+| No log        | 0       | 0    | 2.1184          | 8.3175 | 14.82        | 14.82           | 15.37          |
+| 5.394         | 3.8627  | 50   | 2.1004          | 8.1694 | 14.84        | 14.84           | 31.82          |
+| 4.4484        | 7.7059  | 100  | 2.0367          | 7.6652 | 14.84        | 14.84           | 31.84          |
+| 3.7583        | 11.5490 | 150  | 1.9785          | 7.2316 | 14.84        | 14.84           | 31.84          |
+| 3.363         | 15.3922 | 200  | 1.9299          | 6.8886 | 14.84        | 14.84           | 31.84          |
+| 3.0568        | 19.2353 | 250  | 1.8664          | 6.4652 | 14.84        | 14.84           | 31.84          |
+| 2.8736        | 23.0784 | 300  | 1.8134          | 6.1314 | 14.84        | 14.84           | 31.79          |
+| 2.7646        | 26.9412 | 350  | 1.7851          | 5.9604 | 14.84        | 14.84           | 31.79          |
+| 2.6891        | 30.7843 | 400  | 1.7668          | 5.8523 | 14.84        | 14.84           | 31.79          |
+| 2.6843        | 34.6275 | 450  | 1.7581          | 5.8014 | 14.84        | 14.84           | 31.79          |
+| 2.6048        | 38.4706 | 500  | 1.7534          | 5.7739 | 14.84        | 14.84           | 31.79          |
+| 2.6118        | 42.3137 | 550  | 1.7505          | 5.7573 | 14.84        | 14.84           | 31.79          |
+| 2.6024        | 46.1569 | 600  | 1.7503          | 5.7565 | 14.84        | 14.84           | 31.79          |
+| 2.5727        | 50.0    | 650  | 1.7479          | 5.7428 | 14.84        | 14.84           | 31.79          |
 
 
 ### Framework versions

debug.log

@@ -1846,4 +1846,49 @@ trainable params: 1,572,864 || all params: 3,837,594,624 || trainable%: 0.0410
 
 
                                                                                                   [2026-01-24 15:07:25,861] [INFO] [axolotl.core.trainers.base._save:721] [PID:9359] Saving model checkpoint to ./phi4_african_history_lora_out/checkpoint-650
 
                                                                                                   
-
+
+[2026-01-24 15:07:34,109] [INFO] [axolotl.train.save_trained_model:233] [PID:9359] Training completed! Saving trained model to ./phi4_african_history_lora_out.
+[2026-01-24 15:07:34,468] [INFO] [axolotl.train.save_trained_model:351] [PID:9359] Model successfully saved to ./phi4_african_history_lora_out
+[2026-01-24 15:07:34,702] [INFO] [axolotl.core.trainers.base._save:721] [PID:9359] Saving model checkpoint to ./phi4_african_history_lora_out
+
+
+
+
  ...ora_out/training_args.bin: 100%|█████████████████████████████████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|█████████████████████████████████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|█████████████████████████████████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|█████████████████████████████████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|█████████████████████████████████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|█████████████████████████████████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|█████████████████████████████████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|█████████████████████████████████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|█████████████████████████████████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|█████████████████████████████████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|█████████████████████████████████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|█████████████████████████████████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|█████████████████████████████████| 7.76kB / 7.76kB            
+
  ...adapter_model.safetensors: 100%|█████████████████████████████████| 6.30MB / 6.30MB            
+
  ...y_lora_out/tokenizer.json: 100%|█████████████████████████████████| 15.5MB / 15.5MB