| 2025-04-12 06:50:02,813 INFO __main__: === Starting LoRA fine-tuning === |
| 2025-04-12 06:50:02,813 INFO __main__: Loading tokenizer from: ./models/gemma-2-9b/models--google--gemma-2-9b/snapshots/33c193028431c2fde6c6e51f29e6f17b60cbfac6 |
| 2025-04-12 06:50:03,674 INFO __main__: Loading base model from: ./models/gemma-2-9b/models--google--gemma-2-9b/snapshots/33c193028431c2fde6c6e51f29e6f17b60cbfac6 |
| 2025-04-12 06:50:04,252 INFO accelerate.utils.modeling: We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk). |
| 2025-04-12 06:51:00,547 INFO __main__: Using Gemma2 eager attention. |
| 2025-04-12 06:51:00,547 INFO __main__: Setting up LoRA config... |
| 2025-04-12 06:51:00,547 INFO __main__: Wrapping base model with LoRA adapters... |
| 2025-04-12 06:51:00,988 INFO __main__: Loading training data: ./finetune/data/main/gsm8k_main_train.json |
| 2025-04-12 06:51:01,320 INFO __main__: Loading validation data: ./finetune/data/main/gsm8k_main_test.json |
| 2025-04-12 06:51:01,459 INFO __main__: Tokenizing train dataset... |
| 2025-04-12 06:51:05,395 INFO __main__: Tokenizing val dataset... |
| 2025-04-12 06:51:06,326 INFO __main__: Configuring TrainingArguments for older Transformers... |
| 2025-04-12 06:51:06,355 INFO __main__: Initializing Trainer (with data_collator for labels). |
| 2025-04-12 06:51:06,370 WARNING accelerate.utils.other: Detected kernel version 4.18.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher. |
| 2025-04-12 06:51:06,377 INFO __main__: Starting training... |
| 2025-04-12 08:07:28,153 INFO __main__: Training complete. |
| 2025-04-12 08:07:28,154 INFO __main__: Running manual evaluation on validation set... |
| 2025-04-12 08:09:59,373 INFO __main__: Validation results: {'eval_loss': 0.7726508975028992, 'eval_runtime': 151.2172, 'eval_samples_per_second': 8.723, 'eval_steps_per_second': 1.091, 'epoch': 2.9977251438511976} |
| 2025-04-12 08:09:59,374 INFO __main__: Saving LoRA adapter to: ./finetune/out_lora |
| 2025-04-12 08:09:59,440 INFO __main__: === Done. LoRA adapter saved. === |
|
|
