[2026-05-15 20:47:36,151][evaluator][INFO] - Evaluations stored in the experiment directory: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10
[2026-05-15 20:47:37,469][trainer][INFO] - GradDiff Trainer loaded, output_dir: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10
[2026-05-15 20:47:37,990][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 20:47:37,990][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-0/evals/TOFU_EVAL.json
[2026-05-15 20:47:37,990][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-0/evals/TOFU_SUMMARY.json
[2026-05-15 20:47:39,512][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 20:47:56,288][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 20:48:31,859][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 20:48:31,860][evaluator][INFO] - Result for metric forget_truth_ratio:	0.47428738953801663
[2026-05-15 20:48:31,866][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 20:48:31,867][metrics][INFO] - Evaluating forget_quality
[2026-05-15 20:48:31,867][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 20:48:31,867][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 20:48:33,575][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 20:48:38,682][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.9914018808305264
[2026-05-15 20:48:40,322][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 20:49:38,438][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.9908990311431487
[2026-05-15 20:49:40,107][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 20:49:46,962][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 20:50:03,404][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 20:50:10,398][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 20:50:32,979][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 20:50:34,673][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 20:50:37,298][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 20:50:40,017][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 20:50:41,264][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 20:50:51,579][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 20:50:51,579][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:50:51,579][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 20:50:53,247][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 20:50:55,351][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 20:50:58,818][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 20:51:00,070][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 20:51:17,020][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 20:51:17,020][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:51:17,021][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 20:51:17,021][metrics][INFO] - Evaluating model_utility
[2026-05-15 20:51:17,022][evaluator][INFO] - Result for metric model_utility:	0.627659700025085
[2026-05-15 20:51:19,970][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 20:51:24,417][metrics][INFO] - Evaluating privleak
[2026-05-15 20:51:24,417][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 20:51:24,418][evaluator][INFO] - Result for metric privleak:	-99.92374998001526
[2026-05-15 20:51:25,943][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 20:51:27,209][evaluator][INFO] - Result for metric extraction_strength:	0.9789440160652491
[2026-05-15 20:53:20,445][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 20:53:20,446][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-7/evals/TOFU_EVAL.json
[2026-05-15 20:53:20,446][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-7/evals/TOFU_SUMMARY.json
[2026-05-15 20:53:22,332][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 20:53:27,175][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 20:53:44,945][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 20:53:44,946][evaluator][INFO] - Result for metric forget_truth_ratio:	0.4730894939792442
[2026-05-15 20:53:44,952][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 20:53:44,952][metrics][INFO] - Evaluating forget_quality
[2026-05-15 20:53:44,952][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 20:53:44,952][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 20:53:46,624][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 20:53:50,163][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.9692670270055532
[2026-05-15 20:53:51,411][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 20:54:02,726][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.9524087063257661
[2026-05-15 20:54:04,344][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 20:54:08,987][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 20:54:20,267][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 20:54:25,011][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 20:54:42,147][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 20:54:43,770][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 20:54:45,881][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 20:54:48,041][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 20:54:49,324][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 20:54:50,977][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 20:54:50,978][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:54:50,978][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 20:54:52,310][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 20:54:54,119][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 20:54:56,047][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 20:54:57,257][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 20:54:59,575][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 20:54:59,575][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:54:59,575][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 20:54:59,575][metrics][INFO] - Evaluating model_utility
[2026-05-15 20:54:59,576][evaluator][INFO] - Result for metric model_utility:	0.6196147082442405
[2026-05-15 20:55:02,094][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 20:55:04,561][metrics][INFO] - Evaluating privleak
[2026-05-15 20:55:04,561][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 20:55:04,561][evaluator][INFO] - Result for metric privleak:	-99.73874998005223
[2026-05-15 20:55:05,828][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 20:55:07,109][evaluator][INFO] - Result for metric extraction_strength:	0.9256074087691567
[2026-05-15 20:55:46,772][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 20:55:46,772][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-14/evals/TOFU_EVAL.json
[2026-05-15 20:55:46,772][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-14/evals/TOFU_SUMMARY.json
[2026-05-15 20:55:48,441][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 20:55:53,301][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 20:56:11,080][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 20:56:11,080][evaluator][INFO] - Result for metric forget_truth_ratio:	0.46617108427579196
[2026-05-15 20:56:11,086][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 20:56:11,087][metrics][INFO] - Evaluating forget_quality
[2026-05-15 20:56:11,087][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 20:56:11,087][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 20:56:12,831][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 20:56:16,365][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.8410929980239598
[2026-05-15 20:56:17,697][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 20:56:31,719][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.8132742412207986
[2026-05-15 20:56:33,357][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 20:56:38,029][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 20:56:49,421][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 20:56:54,120][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 20:57:11,248][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 20:57:12,843][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 20:57:15,011][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 20:57:17,172][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 20:57:18,433][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 20:57:20,089][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 20:57:20,089][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:57:20,089][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 20:57:21,664][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 20:57:23,502][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 20:57:25,415][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 20:57:26,640][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 20:57:29,435][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 20:57:29,435][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:57:29,435][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 20:57:29,435][metrics][INFO] - Evaluating model_utility
[2026-05-15 20:57:29,436][evaluator][INFO] - Result for metric model_utility:	0.6099988133731713
[2026-05-15 20:57:32,176][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 20:57:34,633][metrics][INFO] - Evaluating privleak
[2026-05-15 20:57:34,633][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 20:57:34,633][evaluator][INFO] - Result for metric privleak:	-97.40624998051875
[2026-05-15 20:57:35,899][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 20:57:37,228][evaluator][INFO] - Result for metric extraction_strength:	0.7427965259101152
[2026-05-15 20:58:07,096][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 20:58:07,096][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-21/evals/TOFU_EVAL.json
[2026-05-15 20:58:07,096][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-21/evals/TOFU_SUMMARY.json
[2026-05-15 20:58:08,712][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 20:58:13,532][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 20:58:31,342][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 20:58:31,343][evaluator][INFO] - Result for metric forget_truth_ratio:	0.4549261985040715
[2026-05-15 20:58:31,349][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 20:58:31,349][metrics][INFO] - Evaluating forget_quality
[2026-05-15 20:58:31,349][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 20:58:31,349][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 20:58:32,974][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 20:58:36,510][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.6665252035612138
[2026-05-15 20:58:37,768][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 20:58:50,575][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.7048800156683964
[2026-05-15 20:58:52,164][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 20:58:56,799][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 20:59:08,785][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 20:59:13,938][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 20:59:31,046][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 20:59:32,738][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 20:59:34,709][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 20:59:36,871][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 20:59:38,124][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 20:59:42,883][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 20:59:42,883][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:59:42,883][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 20:59:44,109][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 20:59:45,981][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 20:59:47,898][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 20:59:49,143][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 20:59:51,440][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 20:59:51,440][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 20:59:51,440][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 20:59:51,440][metrics][INFO] - Evaluating model_utility
[2026-05-15 20:59:51,441][evaluator][INFO] - Result for metric model_utility:	0.6081851003597214
[2026-05-15 20:59:53,985][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 20:59:56,430][metrics][INFO] - Evaluating privleak
[2026-05-15 20:59:56,430][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 20:59:56,430][evaluator][INFO] - Result for metric privleak:	-93.45874998130824
[2026-05-15 20:59:57,708][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 20:59:59,090][evaluator][INFO] - Result for metric extraction_strength:	0.5560347275460378
[2026-05-15 21:00:27,402][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 21:00:27,403][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-28/evals/TOFU_EVAL.json
[2026-05-15 21:00:27,403][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-28/evals/TOFU_SUMMARY.json
[2026-05-15 21:00:29,075][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 21:00:33,923][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 21:00:51,709][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 21:00:51,709][evaluator][INFO] - Result for metric forget_truth_ratio:	0.4387997327040273
[2026-05-15 21:00:51,715][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 21:00:51,716][metrics][INFO] - Evaluating forget_quality
[2026-05-15 21:00:51,716][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 21:00:51,716][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 21:00:53,381][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 21:00:56,914][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.5035118470614305
[2026-05-15 21:00:58,202][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 21:01:09,828][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.6451349587675484
[2026-05-15 21:01:11,544][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 21:01:16,148][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 21:01:27,670][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 21:01:32,382][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 21:01:49,504][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 21:01:51,173][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 21:01:53,158][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 21:01:55,319][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 21:01:56,565][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 21:01:59,049][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 21:01:59,049][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:01:59,049][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 21:02:00,288][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 21:02:02,220][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 21:02:04,134][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 21:02:05,389][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 21:02:07,823][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 21:02:07,823][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:02:07,823][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 21:02:07,823][metrics][INFO] - Evaluating model_utility
[2026-05-15 21:02:07,824][evaluator][INFO] - Result for metric model_utility:	0.5965473302371095
[2026-05-15 21:02:10,333][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 21:02:12,779][metrics][INFO] - Evaluating privleak
[2026-05-15 21:02:12,780][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 21:02:12,780][evaluator][INFO] - Result for metric privleak:	-88.71874998225626
[2026-05-15 21:02:14,428][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 21:02:15,859][evaluator][INFO] - Result for metric extraction_strength:	0.4216744483561699
[2026-05-15 21:02:44,259][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 21:02:44,259][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-35/evals/TOFU_EVAL.json
[2026-05-15 21:02:44,259][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-35/evals/TOFU_SUMMARY.json
[2026-05-15 21:02:45,895][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 21:02:50,726][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 21:03:08,508][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 21:03:08,509][evaluator][INFO] - Result for metric forget_truth_ratio:	0.4202630036382324
[2026-05-15 21:03:08,515][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 21:03:08,516][metrics][INFO] - Evaluating forget_quality
[2026-05-15 21:03:08,516][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 21:03:08,516][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 21:03:10,225][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 21:03:13,760][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.40284690549328445
[2026-05-15 21:03:15,015][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 21:03:26,218][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.605656249819598
[2026-05-15 21:03:27,998][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 21:03:32,650][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 21:03:44,191][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 21:03:48,924][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 21:04:06,039][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 21:04:07,652][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 21:04:09,627][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 21:04:11,794][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 21:04:13,173][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 21:04:15,956][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 21:04:15,956][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:04:15,956][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 21:04:17,198][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 21:04:19,042][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 21:04:20,957][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 21:04:22,184][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 21:04:28,626][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 21:04:28,626][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:04:28,626][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 21:04:28,627][metrics][INFO] - Evaluating model_utility
[2026-05-15 21:04:28,627][evaluator][INFO] - Result for metric model_utility:	0.5919050061738889
[2026-05-15 21:04:31,528][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 21:04:33,983][metrics][INFO] - Evaluating privleak
[2026-05-15 21:04:33,983][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 21:04:33,983][evaluator][INFO] - Result for metric privleak:	-85.04124998299176
[2026-05-15 21:04:35,216][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 21:04:36,657][evaluator][INFO] - Result for metric extraction_strength:	0.3655469828538317
[2026-05-15 21:05:02,092][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 21:05:02,092][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-42/evals/TOFU_EVAL.json
[2026-05-15 21:05:02,092][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-42/evals/TOFU_SUMMARY.json
[2026-05-15 21:05:03,786][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 21:05:08,663][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 21:05:26,432][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 21:05:26,433][evaluator][INFO] - Result for metric forget_truth_ratio:	0.40873454402250364
[2026-05-15 21:05:26,439][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 21:05:26,439][metrics][INFO] - Evaluating forget_quality
[2026-05-15 21:05:26,439][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 21:05:26,440][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 21:05:28,084][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 21:05:31,619][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.3475085932298115
[2026-05-15 21:05:32,854][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 21:05:46,701][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.5880500988235582
[2026-05-15 21:05:48,413][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 21:05:53,056][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 21:06:04,666][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 21:06:09,345][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 21:06:26,487][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 21:06:28,096][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 21:06:30,090][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 21:06:32,252][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 21:06:33,489][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 21:06:36,166][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 21:06:36,166][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:06:36,166][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 21:06:37,388][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 21:06:39,293][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 21:06:41,206][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 21:06:42,440][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 21:06:44,814][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 21:06:44,814][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:06:44,814][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 21:06:44,814][metrics][INFO] - Evaluating model_utility
[2026-05-15 21:06:44,815][evaluator][INFO] - Result for metric model_utility:	0.5880946216453223
[2026-05-15 21:06:47,581][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 21:06:50,027][metrics][INFO] - Evaluating privleak
[2026-05-15 21:06:50,027][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 21:06:50,027][evaluator][INFO] - Result for metric privleak:	-82.59374998348122
[2026-05-15 21:06:51,253][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 21:06:52,699][evaluator][INFO] - Result for metric extraction_strength:	0.34268577632141345
[2026-05-15 21:07:22,298][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 21:07:22,298][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-49/evals/TOFU_EVAL.json
[2026-05-15 21:07:22,298][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-49/evals/TOFU_SUMMARY.json
[2026-05-15 21:07:23,905][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 21:07:28,741][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 21:07:46,522][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 21:07:46,523][evaluator][INFO] - Result for metric forget_truth_ratio:	0.4040336600608835
[2026-05-15 21:07:46,529][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 21:07:46,529][metrics][INFO] - Evaluating forget_quality
[2026-05-15 21:07:46,529][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 21:07:46,529][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 21:07:48,192][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 21:07:51,726][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.3216440254848527
[2026-05-15 21:07:52,950][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 21:08:04,325][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.5804104561740604
[2026-05-15 21:08:05,947][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 21:08:10,538][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 21:08:22,182][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 21:08:26,980][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 21:08:44,103][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 21:08:45,772][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 21:08:47,743][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 21:08:49,906][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 21:08:51,222][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 21:08:53,716][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 21:08:53,716][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:08:53,716][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 21:08:55,032][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 21:08:56,840][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 21:08:58,759][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 21:09:00,016][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 21:09:02,396][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 21:09:02,396][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:09:02,396][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 21:09:02,396][metrics][INFO] - Evaluating model_utility
[2026-05-15 21:09:02,397][evaluator][INFO] - Result for metric model_utility:	0.5851194256837976
[2026-05-15 21:09:04,895][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 21:09:07,342][metrics][INFO] - Evaluating privleak
[2026-05-15 21:09:07,343][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 21:09:07,343][evaluator][INFO] - Result for metric privleak:	-80.86499998382699
[2026-05-15 21:09:08,696][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 21:09:10,143][evaluator][INFO] - Result for metric extraction_strength:	0.32749478043586594
[2026-05-15 21:09:35,104][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 21:09:35,104][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-56/evals/TOFU_EVAL.json
[2026-05-15 21:09:35,104][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-56/evals/TOFU_SUMMARY.json
[2026-05-15 21:09:36,755][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 21:09:41,634][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 21:09:59,409][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 21:09:59,410][evaluator][INFO] - Result for metric forget_truth_ratio:	0.40082006515227664
[2026-05-15 21:09:59,416][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 21:09:59,416][metrics][INFO] - Evaluating forget_quality
[2026-05-15 21:09:59,416][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 21:09:59,416][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 21:10:01,155][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 21:10:04,700][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.3120441889709665
[2026-05-15 21:10:05,965][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 21:10:18,061][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.5754668877083341
[2026-05-15 21:10:19,733][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 21:10:24,334][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 21:10:35,916][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 21:10:40,715][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 21:10:57,846][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 21:10:59,470][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 21:11:01,445][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 21:11:03,622][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 21:11:04,837][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 21:11:07,099][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 21:11:07,099][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:11:07,099][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 21:11:08,381][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 21:11:10,232][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 21:11:12,149][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 21:11:13,458][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 21:11:15,846][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 21:11:15,846][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:11:15,846][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 21:11:15,846][metrics][INFO] - Evaluating model_utility
[2026-05-15 21:11:15,847][evaluator][INFO] - Result for metric model_utility:	0.5856610431800201
[2026-05-15 21:11:18,333][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 21:11:20,787][metrics][INFO] - Evaluating privleak
[2026-05-15 21:11:20,787][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 21:11:20,787][evaluator][INFO] - Result for metric privleak:	-80.17374998396525
[2026-05-15 21:11:22,046][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 21:11:23,507][evaluator][INFO] - Result for metric extraction_strength:	0.3196129620730693
[2026-05-15 21:11:41,580][evaluator][INFO] - ***** Running TOFU evaluation suite *****
[2026-05-15 21:11:41,580][evaluator][INFO] - Fine-grained evaluations will be saved to: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-60/evals/TOFU_EVAL.json
[2026-05-15 21:11:41,580][evaluator][INFO] - Aggregated evaluations will be summarised in: ./saves/unlearn/C_C1_GradDiff_NoQAT_tofu_Llama-3.1-8B-Instruct_forget10/checkpoint-60/evals/TOFU_SUMMARY.json
[2026-05-15 21:11:43,248][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
[2026-05-15 21:11:48,261][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
[2026-05-15 21:12:06,078][metrics][INFO] - Evaluating forget_truth_ratio
[2026-05-15 21:12:06,078][evaluator][INFO] - Result for metric forget_truth_ratio:	0.4010539704765916
[2026-05-15 21:12:06,085][metrics][INFO] - Skipping forget_quality's precompute forget_truth_ratio, already evaluated.
[2026-05-15 21:12:06,085][metrics][INFO] - Evaluating forget_quality
[2026-05-15 21:12:06,085][metrics][WARNING] - retain_model_logs not provided in reference_logs, setting forget_quality to None
[2026-05-15 21:12:06,085][evaluator][INFO] - Result for metric forget_quality:	None
[2026-05-15 21:12:07,911][metrics][INFO] - Evaluating forget_Q_A_Prob
[2026-05-15 21:12:11,456][evaluator][INFO] - Result for metric forget_Q_A_Prob:	0.3111993143450383
[2026-05-15 21:12:12,800][metrics][INFO] - Evaluating forget_Q_A_ROUGE
[2026-05-15 21:12:24,418][evaluator][INFO] - Result for metric forget_Q_A_ROUGE:	0.5735587212059887
[2026-05-15 21:12:26,037][metrics][INFO] - Evaluating retain_Q_A_Prob
[2026-05-15 21:12:31,573][metrics][INFO] - Evaluating retain_Q_A_ROUGE
[2026-05-15 21:12:43,223][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
[2026-05-15 21:12:47,974][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
[2026-05-15 21:13:05,100][metrics][INFO] - Evaluating retain_Truth_Ratio
[2026-05-15 21:13:06,776][metrics][INFO] - Evaluating ra_Q_A_Prob
[2026-05-15 21:13:08,775][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
[2026-05-15 21:13:10,942][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
[2026-05-15 21:13:12,195][metrics][INFO] - Evaluating ra_Q_A_ROUGE
[2026-05-15 21:13:14,874][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
[2026-05-15 21:13:14,874][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:13:14,874][metrics][INFO] - Evaluating ra_Truth_Ratio
[2026-05-15 21:13:16,143][metrics][INFO] - Evaluating wf_Q_A_Prob
[2026-05-15 21:13:17,974][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
[2026-05-15 21:13:19,888][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
[2026-05-15 21:13:21,135][metrics][INFO] - Evaluating wf_Q_A_ROUGE
[2026-05-15 21:13:23,527][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
[2026-05-15 21:13:23,528][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
[2026-05-15 21:13:23,528][metrics][INFO] - Evaluating wf_Truth_Ratio
[2026-05-15 21:13:23,528][metrics][INFO] - Evaluating model_utility
[2026-05-15 21:13:23,528][evaluator][INFO] - Result for metric model_utility:	0.5843879678433459
[2026-05-15 21:13:26,397][metrics][INFO] - Evaluating mia_min_k
[2026-05-15 21:13:28,846][metrics][INFO] - Evaluating privleak
[2026-05-15 21:13:28,846][metrics][WARNING] - retain_model_logs evals not provided for privleak, using default retain auc of 0.5
[2026-05-15 21:13:28,846][evaluator][INFO] - Result for metric privleak:	-80.13374998397327
[2026-05-15 21:13:30,191][metrics][INFO] - Evaluating extraction_strength
[2026-05-15 21:13:31,644][evaluator][INFO] - Result for metric extraction_strength:	0.32156534302545026