End of training

README.md

@@ -0,0 +1,143 @@
+---
+library_name: peft
+license: mit
+base_model: microsoft/Phi-4-mini-instruct
+tags:
+- axolotl
+- base_model:adapter:microsoft/Phi-4-mini-instruct
+- lora
+- transformers
+datasets:
+- DannyAI/African-History-QA-Dataset
+pipeline_tag: text-generation
+model-index:
+- name: phi4_lora_axolotl
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
+<details><summary>See axolotl config</summary>
+
+axolotl version: `0.14.0.dev0`
+```yaml
+base_model: microsoft/Phi-4-mini-instruct
+model_type: AutoModelForCausalLM
+tokenizer_type: AutoTokenizer
+
+# 1. Dataset Configuration
+datasets:
+  - path: DannyAI/African-History-QA-Dataset
+    split: train
+    type: alpaca_chat.load_qa
+    system_prompt: "You are a helpful AI assistant specialised in African history which gives concise answers to questions asked"
+test_datasets:
+  - path: DannyAI/African-History-QA-Dataset
+    split: validation
+    type: alpaca_chat.load_qa
+    system_prompt: "You are a helpful AI assistant specialised in African history which gives concise answers to questions asked"
+
+# 2. Output & Chat Configuration
+output_dir: ./phi4_african_history_lora_out
+chat_template: tokenizer_default
+train_on_inputs: false
+
+# 3. Batch Size Configuration
+micro_batch_size: 2
+gradient_accumulation_steps: 4
+
+# 4. LoRA Configuration
+adapter: lora
+lora_r: 8
+lora_alpha: 16
+lora_dropout: 0.05
+lora_target_modules: [q_proj, v_proj, k_proj, o_proj]
+
+# 5. Hardware & Efficiency
+sequence_len: 2048
+sample_packing: true
+eval_sample_packing: false 
+pad_to_sequence_len: true
+bf16: true
+fp16: false
+
+# 6. Training Duration & Optimizer
+num_epochs: 10
+warmup_steps: 10
+learning_rate: 0.00002
+optimizer: adamw_torch 
+lr_scheduler: cosine
+
+# 7. Logging & Evaluation
+wandb_project: phi4_african_history
+wandb_name: phi4_lora_axolotl
+eval_strategy: steps
+eval_steps: 100
+save_strategy: steps
+save_steps: 200
+logging_steps: 5
+
+# 8. Public Hugging Face Hub Upload
+hub_model_id: DannyAI/phi4_lora_axolotl
+push_adapter_to_hub: true
+hub_private_repo: false
+
+```
+
+</details><br>
+
+# phi4_lora_axolotl
+
+This model is a fine-tuned version of [microsoft/Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) on the DannyAI/African-History-QA-Dataset dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.0938
+- Ppl: 8.1156
+- Memory/max Active (gib): 14.84
+- Memory/max Allocated (gib): 14.84
+- Memory/device Reserved (gib): 31.82
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 8
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10
+- training_steps: 120
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Ppl    | Active (gib) | Allocated (gib) | Reserved (gib) |
+|:-------------:|:------:|:----:|:---------------:|:------:|:------------:|:---------------:|:--------------:|
+| No log        | 0      | 0    | 2.1335          | 8.4442 | 14.82        | 14.82           | 15.37          |
+| 4.9679        | 7.7059 | 100  | 2.0938          | 8.1156 | 14.84        | 14.84           | 31.82          |
+
+
+### Framework versions
+
+- PEFT 0.18.1
+- Transformers 4.57.6
+- Pytorch 2.9.1+cu128
+- Datasets 4.5.0
+- Tokenizers 0.22.2

debug.log

@@ -473,4 +473,49 @@ trainable params: 1,572,864 || all params: 3,837,594,624 || trainable%: 0.0410
 
 96%|████████████████████████████████▌ | 115/120 [17:35<00:41,  8.38s/it]
 97%|████████████████████████████████▊ | 116/120 [17:43<00:33,  8.34s/it]
 98%|█████████████████████████████████▏| 117/120 [17:49<00:23,  7.72s/it]
 98%|█████████████████████████████████▍| 118/120 [17:59<00:16,  8.47s/it]
 99%|█████████████████████████████████▋| 119/120 [18:08<00:08,  8.41s/it]
                                                                         
 
 
                                                                         
-
+
+[2026-01-24 13:11:17,545] [INFO] [axolotl.train.save_trained_model:233] [PID:6937] Training completed! Saving trained model to ./phi4_african_history_lora_out.
+[2026-01-24 13:11:17,908] [INFO] [axolotl.train.save_trained_model:351] [PID:6937] Model successfully saved to ./phi4_african_history_lora_out
+[2026-01-24 13:11:18,131] [INFO] [axolotl.core.trainers.base._save:721] [PID:6937] Saving model checkpoint to ./phi4_african_history_lora_out
+
+
+
+
  ...ora_out/training_args.bin: 100%|███████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|███████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|███████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|███████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|███████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|███████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|███████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|███████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|███████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|███████| 7.76kB / 7.76kB            
+
+
+
  ...adapter_model.safetensors: 100%|███████| 6.30MB / 6.30MB            
+
+
+
+
  ...y_lora_out/tokenizer.json: 100%|███████| 15.5MB / 15.5MB            
+
+
  ...ora_out/training_args.bin: 100%|███████| 7.76kB / 7.76kB            
+
  ...adapter_model.safetensors: 100%|███████| 6.30MB / 6.30MB            
+
  ...y_lora_out/tokenizer.json: 100%|███████| 15.5MB / 15.5MB