source/eval/outputs/reeval_run.log

2026-03-05 10:57:53 [INFO] ========================================================================
2026-03-05 10:57:53 [INFO]   FRANKENSTALLM 3B — Re-evaluation Pipeline
2026-03-05 10:57:53 [INFO] ========================================================================
2026-03-05 10:57:53 [INFO]   Previous run : eval/outputs/3b_full_eval_20260305_0318
2026-03-05 10:57:53 [INFO]   HF checkpoint: eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
2026-03-05 10:57:53 [INFO]   Output dir   : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057
2026-03-05 10:57:53 [INFO]   GPUs         : [0, 1, 2, 3, 4, 5, 6, 7]
2026-03-05 10:57:53 [INFO]   Skip Phase 2 : False
2026-03-05 10:57:53 [INFO] 
2026-03-05 10:57:53 [INFO] ------------------------------------------------------------------------
2026-03-05 10:57:53 [INFO]   PHASE 1 — Copy from Previous Run
2026-03-05 10:57:53 [INFO] ------------------------------------------------------------------------
2026-03-05 10:57:53 [INFO]   Copied phase1_results.json from previous run
2026-03-05 10:57:53 [INFO]   Copied generation_samples.json
2026-03-05 10:57:53 [INFO] 
2026-03-05 10:57:53 [INFO] ------------------------------------------------------------------------
2026-03-05 10:57:53 [INFO]   PHASE 2 — Corrected Benchmarks — 8 GPU Parallel
2026-03-05 10:57:53 [INFO] ------------------------------------------------------------------------
2026-03-05 10:57:53 [INFO]   Running 0-shot benchmarks on 8 GPUs ...
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 0 — KoBEST: boolq + copa (GPU 0)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 1 — KoBEST: hellaswag + sentineg (GPU 1)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 2 — KoBEST: wic (GPU 2)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 3 — HAE-RAE (all subtasks) (GPU 3)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 4 — MMLU-KO (57 subtasks) (GPU 4)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 5 — EN: hellaswag + arc_easy + arc_challenge (GPU 5)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 6 — EN: winogrande + piqa (GPU 6)
2026-03-05 10:57:53 [INFO]   Spawning: [0-shot] GPU 7 — MMLU-EN (57 subtasks) (GPU 7)
2026-03-05 10:59:21 [INFO]   [DONE] [0-shot] GPU 2 — KoBEST: wic
2026-03-05 10:59:33 [INFO]   [DONE] [0-shot] GPU 1 — KoBEST: hellaswag + sentineg
2026-03-05 10:59:35 [INFO]   [DONE] [0-shot] GPU 4 — MMLU-KO (57 subtasks)
2026-03-05 10:59:41 [INFO]   [DONE] [0-shot] GPU 0 — KoBEST: boolq + copa
2026-03-05 10:59:45 [INFO]   [DONE] [0-shot] GPU 3 — HAE-RAE (all subtasks)
2026-03-05 10:59:47 [INFO]   [DONE] [0-shot] GPU 6 — EN: winogrande + piqa
2026-03-05 11:00:41 [INFO]   [DONE] [0-shot] GPU 5 — EN: hellaswag + arc_easy + arc_challenge
2026-03-05 11:01:30 [INFO]   [DONE] [0-shot] GPU 7 — MMLU-EN (57 subtasks)
2026-03-05 11:01:30 [INFO]   Complete: 8 succeeded, 0 failed
2026-03-05 11:01:30 [INFO]   0-shot complete.
2026-03-05 11:01:31 [INFO]   Running 5-shot benchmarks (Korean only) ...
2026-03-05 11:01:31 [INFO]   Spawning: [5-shot] GPU 0 — KoBEST: boolq + copa (GPU 0)
2026-03-05 11:01:31 [INFO]   Spawning: [5-shot] GPU 1 — KoBEST: hellaswag + sentineg (GPU 1)
2026-03-05 11:01:31 [INFO]   Spawning: [5-shot] GPU 2 — KoBEST: wic (GPU 2)
2026-03-05 11:01:31 [INFO]   Spawning: [5-shot] GPU 3 — HAE-RAE (all subtasks) (GPU 3)
2026-03-05 11:01:31 [INFO]   Spawning: [5-shot] GPU 4 — MMLU-KO (57 subtasks) (GPU 4)
2026-03-05 11:02:07 [INFO]   [DONE] [5-shot] GPU 0 — KoBEST: boolq + copa
2026-03-05 11:02:31 [INFO]   [DONE] [5-shot] GPU 2 — KoBEST: wic
2026-03-05 11:02:33 [INFO]   [DONE] [5-shot] GPU 4 — MMLU-KO (57 subtasks)
2026-03-05 11:02:40 [INFO]   [DONE] [5-shot] GPU 1 — KoBEST: hellaswag + sentineg
2026-03-05 11:02:58 [INFO]   [DONE] [5-shot] GPU 3 — HAE-RAE (all subtasks)
2026-03-05 11:02:58 [INFO]   Complete: 5 succeeded, 0 failed
2026-03-05 11:02:58 [INFO]   5-shot complete.
2026-03-05 11:03:00 [INFO]   Phase 2 results saved: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/phase2_results.json
2026-03-05 11:03:00 [INFO]   Phase 2 complete in 5m 6s.
2026-03-05 11:03:00 [INFO] 
2026-03-05 11:03:00 [INFO] ------------------------------------------------------------------------
2026-03-05 11:03:00 [INFO]   PHASE 3 — Report Generation (Fixed)
2026-03-05 11:03:00 [INFO] ------------------------------------------------------------------------
2026-03-05 11:03:00 [INFO]   Report saved: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/full_eval_report.md
2026-03-05 11:03:00 [INFO]   Individual reports: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/reports
2026-03-05 11:03:00 [INFO]   Phase 3 complete in 0s.
2026-03-05 11:03:00 [INFO] ========================================================================
2026-03-05 11:03:00 [INFO]   RE-EVALUATION COMPLETE
2026-03-05 11:03:00 [INFO] ========================================================================
2026-03-05 11:03:00 [INFO]   Total time      : 5m 6s
2026-03-05 11:03:00 [INFO]   Output dir      : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057
2026-03-05 11:03:00 [INFO]   Phase 1 results : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/phase1_results.json
2026-03-05 11:03:00 [INFO]   Phase 2 results : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/phase2_results.json
2026-03-05 11:03:00 [INFO]   Report          : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/full_eval_report.md
2026-03-05 11:03:00 [INFO]   Reports dir     : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_reeval_20260305_1057/reports
2026-03-05 11:03:00 [INFO]   Phase 2 (0-shot): 8 OK / 0 failed
2026-03-05 11:03:00 [INFO]   Phase 2 (5-shot): 5 OK
2026-03-05 11:03:00 [INFO] ========================================================================