Upload folder using huggingface_hub

logs/eval_gpu_20251130_084524.log

@@ -0,0 +1,35 @@
+2025-11-30 08:45:24 - eval_gpu - INFO - Logging to: logs/codet5/eval_gpu_20251130_084524.log
+2025-11-30 08:45:24 - eval_gpu - INFO - Monitor progress: tail -f logs/codet5/eval_gpu_20251130_084524.log
+2025-11-30 08:45:24 - eval_gpu - INFO - ============================================================
+2025-11-30 08:45:24 - eval_gpu - INFO - CodeT5+ Evaluation with GPU Support
+2025-11-30 08:45:24 - eval_gpu - INFO - ============================================================
+2025-11-30 08:45:24 - eval_gpu - INFO - Using GPU: NVIDIA GeForce RTX 5090
+2025-11-30 08:45:24 - eval_gpu - INFO - Configuration:
+2025-11-30 08:45:24 - eval_gpu - INFO -   checkpoint: model/checkpoints/run1-java-codet5
+2025-11-30 08:45:24 - eval_gpu - INFO -   data: datasets/java
+2025-11-30 08:45:24 - eval_gpu - INFO -   k: 5
+2025-11-30 08:45:24 - eval_gpu - INFO -   device: cuda
+2025-11-30 08:45:24 - eval_gpu - INFO -   batch_size: 16
+2025-11-30 08:45:25 - eval_gpu - INFO - Model and tokenizer loaded
+2025-11-30 08:45:25 - eval_gpu - INFO - Model moved to cuda and set to eval mode
+2025-11-30 08:45:26 - eval_gpu - INFO - Loaded 34496 test examples
+2025-11-30 08:45:26 - eval_gpu - INFO - Starting evaluation...
+2025-11-30 08:45:26 - eval_gpu - INFO - Total batches: 2156
+2025-11-30 09:04:57 - eval_gpu - INFO - Evaluation completed in 1172.85 seconds
+2025-11-30 09:04:57 - eval_gpu - INFO - Average speed: 29.41 samples/second
+2025-11-30 09:04:57 - eval_gpu - INFO - Metrics saved to: model/metrics/run1-java-codet5/metrics.json
+2025-11-30 09:04:58 - eval_gpu - INFO - Detailed results saved to: model/metrics/run1-java-codet5/detailed_results.jsonl
+2025-11-30 09:04:58 - eval_gpu - INFO - ============================================================
+2025-11-30 09:04:58 - eval_gpu - INFO - EVALUATION RESULTS
+2025-11-30 09:04:58 - eval_gpu - INFO - ============================================================
+2025-11-30 09:04:58 - eval_gpu - INFO -   n: 34496
+2025-11-30 09:04:58 - eval_gpu - INFO -   exact_match: 0.6631
+2025-11-30 09:04:58 - eval_gpu - INFO -   exact_match_case_insensitive: 0.6650
+2025-11-30 09:04:58 - eval_gpu - INFO -   topk_accuracy: 0.7395
+2025-11-30 09:04:58 - eval_gpu - INFO -   avg_levenshtein: 3.7492
+2025-11-30 09:04:58 - eval_gpu - INFO -   k: 5
+2025-11-30 09:04:58 - eval_gpu - INFO -   device: cuda
+2025-11-30 09:04:58 - eval_gpu - INFO -   batch_size: 16
+2025-11-30 09:04:58 - eval_gpu - INFO -   elapsed_time_seconds: 1172.8496
+2025-11-30 09:04:58 - eval_gpu - INFO -   samples_per_second: 29.4121
+2025-11-30 09:04:58 - eval_gpu - INFO - Total evaluation time: 1172.85 seconds (19.55 minutes)

logs/train_20251129_215753.log

@@ -0,0 +1,38 @@
+2025-11-29 21:57:53 - train - INFO - Logging to: logs/codet5/train_20251129_215753.log
+2025-11-29 21:57:53 - train - INFO - Monitor progress: tail -f logs/codet5/train_20251129_215753.log
+2025-11-29 21:57:53 - train - INFO - ============================================================
+2025-11-29 21:57:53 - train - INFO - CodeT5+ Training
+2025-11-29 21:57:53 - train - INFO - ============================================================
+2025-11-29 21:57:53 - train - INFO - Using CUDA device: 0
+2025-11-29 21:57:53 - train - INFO - GPU: NVIDIA GeForce RTX 5090
+2025-11-29 21:57:53 - train - INFO - Configuration:
+2025-11-29 21:57:53 - train - INFO -   model: Salesforce/codet5p-220m
+2025-11-29 21:57:53 - train - INFO -   data: datasets/java
+2025-11-29 21:57:53 - train - INFO -   output: model/checkpoints/run1-java-codet5
+2025-11-29 21:57:53 - train - INFO -   batch_size: 10
+2025-11-29 21:57:53 - train - INFO -   gradient_accumulation_steps: 4
+2025-11-29 21:57:53 - train - INFO -   effective_batch_size: 40
+2025-11-29 21:57:53 - train - INFO -   learning_rate: 5e-05
+2025-11-29 21:57:53 - train - INFO -   epochs: 5
+2025-11-29 21:57:53 - train - INFO -   max_source_len: 1024
+2025-11-29 21:57:53 - train - INFO -   max_target_len: 32
+2025-11-29 21:57:53 - train - INFO -   fp16: True
+2025-11-29 21:57:53 - train - INFO -   seed: 42
+2025-11-29 21:57:53 - train - INFO - Loading tokenizer and model...
+2025-11-29 21:58:04 - train - INFO - Model loaded: Salesforce/codet5p-220m
+2025-11-29 21:58:04 - train - INFO - Loading and preprocessing dataset...
+2025-11-29 21:58:06 - train - INFO - Train examples: 275962
+2025-11-29 21:58:06 - train - INFO - Validation examples: 34495
+2025-11-29 22:08:37 - train - INFO - Dataset preprocessing completed
+2025-11-29 22:08:37 - train - INFO - Starting training...
+2025-11-29 22:08:37 - train - INFO - Total training steps: 34495
+2025-11-29 22:08:37 - train - INFO - No checkpoint found for auto-resume, starting from scratch
+2025-11-30 08:45:17 - train - INFO - Training completed in 38843.71 seconds (10.79 hours)
+2025-11-30 08:45:17 - train - INFO - Saving model to model/checkpoints/run1-java-codet5
+2025-11-30 08:45:18 - train - INFO - Model and tokenizer saved successfully
+2025-11-30 08:45:18 - train - INFO - ============================================================
+2025-11-30 08:45:18 - train - INFO - Training Summary
+2025-11-30 08:45:18 - train - INFO - ============================================================
+2025-11-30 08:45:18 - train - INFO - Total time: 10.79 hours
+2025-11-30 08:45:18 - train - INFO - Output directory: model/checkpoints/run1-java-codet5
+2025-11-30 08:45:18 - train - INFO - Training log: model/checkpoints/run1-java-codet5/training_log.csv