| [2026-06-11 21:10:37,292][huggingface_hub.utils._http][WARNING] - Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. |
| [2026-06-11 21:18:27,436][trainer][INFO] - GradDiff Trainer loaded, output_dir: ./saves/unlearn/multilingual/books.all/chapters.all/GradDiff.KL/de_alpha_1000 |
| [2026-06-11 21:59:33,010][lm_eval.models.huggingface][WARNING] - `pretrained` model kwarg is not of type `str`. Many other model arguments may be ignored. Please do not launch via accelerate or use `parallelize=True` if passing an existing model this way. |
| [2026-06-11 21:59:34,854][lm_eval.models.huggingface][WARNING] - Passed an already-initialized model through `pretrained`, assuming single-process call to evaluate() or custom distributed integration |
| [2026-06-11 21:59:34,868][lm_eval.evaluator][INFO] - Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234 |
| [2026-06-11 21:59:34,868][lm_eval.evaluator][INFO] - Using pre-initialized model |
| [2026-06-11 22:01:26,514][lm_eval.evaluator_utils][INFO] - Selected tasks: |
| [2026-06-11 22:01:26,517][lm_eval.evaluator_utils][INFO] - Group: mmlu |
| [2026-06-11 22:01:26,519][lm_eval.evaluator_utils][INFO] - Group: mmlu_stem |
| [2026-06-11 22:01:26,520][lm_eval.evaluator_utils][INFO] - Task: mmlu_abstract_algebra (mmlu/default/mmlu_abstract_algebra.yaml) |
| [2026-06-11 22:01:26,521][lm_eval.evaluator_utils][INFO] - Task: mmlu_anatomy (mmlu/default/mmlu_anatomy.yaml) |
| [2026-06-11 22:01:26,522][lm_eval.evaluator_utils][INFO] - Task: mmlu_astronomy (mmlu/default/mmlu_astronomy.yaml) |
| [2026-06-11 22:01:26,523][lm_eval.evaluator_utils][INFO] - Task: mmlu_college_biology (mmlu/default/mmlu_college_biology.yaml) |
| [2026-06-11 22:01:26,524][lm_eval.evaluator_utils][INFO] - Task: mmlu_college_chemistry (mmlu/default/mmlu_college_chemistry.yaml) |
| [2026-06-11 22:01:26,525][lm_eval.evaluator_utils][INFO] - Task: mmlu_college_computer_science (mmlu/default/mmlu_college_computer_science.yaml) |
| [2026-06-11 22:01:26,526][lm_eval.evaluator_utils][INFO] - Task: mmlu_college_mathematics (mmlu/default/mmlu_college_mathematics.yaml) |
| [2026-06-11 22:01:26,527][lm_eval.evaluator_utils][INFO] - Task: mmlu_college_physics (mmlu/default/mmlu_college_physics.yaml) |
| [2026-06-11 22:01:26,529][lm_eval.evaluator_utils][INFO] - Task: mmlu_computer_security (mmlu/default/mmlu_computer_security.yaml) |
| [2026-06-11 22:01:26,529][lm_eval.evaluator_utils][INFO] - Task: mmlu_conceptual_physics (mmlu/default/mmlu_conceptual_physics.yaml) |
| [2026-06-11 22:01:26,530][lm_eval.evaluator_utils][INFO] - Task: mmlu_electrical_engineering (mmlu/default/mmlu_electrical_engineering.yaml) |
| [2026-06-11 22:01:26,531][lm_eval.evaluator_utils][INFO] - Task: mmlu_elementary_mathematics (mmlu/default/mmlu_elementary_mathematics.yaml) |
| [2026-06-11 22:01:26,532][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_biology (mmlu/default/mmlu_high_school_biology.yaml) |
| [2026-06-11 22:01:26,533][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_chemistry (mmlu/default/mmlu_high_school_chemistry.yaml) |
| [2026-06-11 22:01:26,534][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_computer_science (mmlu/default/mmlu_high_school_computer_science.yaml) |
| [2026-06-11 22:01:26,535][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_mathematics (mmlu/default/mmlu_high_school_mathematics.yaml) |
| [2026-06-11 22:01:26,536][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_physics (mmlu/default/mmlu_high_school_physics.yaml) |
| [2026-06-11 22:01:26,537][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_statistics (mmlu/default/mmlu_high_school_statistics.yaml) |
| [2026-06-11 22:01:26,538][lm_eval.evaluator_utils][INFO] - Task: mmlu_machine_learning (mmlu/default/mmlu_machine_learning.yaml) |
| [2026-06-11 22:01:26,538][lm_eval.evaluator_utils][INFO] - Group: mmlu_other |
| [2026-06-11 22:01:26,539][lm_eval.evaluator_utils][INFO] - Task: mmlu_business_ethics (mmlu/default/mmlu_business_ethics.yaml) |
| [2026-06-11 22:01:26,540][lm_eval.evaluator_utils][INFO] - Task: mmlu_clinical_knowledge (mmlu/default/mmlu_clinical_knowledge.yaml) |
| [2026-06-11 22:01:26,541][lm_eval.evaluator_utils][INFO] - Task: mmlu_college_medicine (mmlu/default/mmlu_college_medicine.yaml) |
| [2026-06-11 22:01:26,542][lm_eval.evaluator_utils][INFO] - Task: mmlu_global_facts (mmlu/default/mmlu_global_facts.yaml) |
| [2026-06-11 22:01:26,543][lm_eval.evaluator_utils][INFO] - Task: mmlu_human_aging (mmlu/default/mmlu_human_aging.yaml) |
| [2026-06-11 22:01:26,544][lm_eval.evaluator_utils][INFO] - Task: mmlu_management (mmlu/default/mmlu_management.yaml) |
| [2026-06-11 22:01:26,545][lm_eval.evaluator_utils][INFO] - Task: mmlu_marketing (mmlu/default/mmlu_marketing.yaml) |
| [2026-06-11 22:01:26,546][lm_eval.evaluator_utils][INFO] - Task: mmlu_medical_genetics (mmlu/default/mmlu_medical_genetics.yaml) |
| [2026-06-11 22:01:26,546][lm_eval.evaluator_utils][INFO] - Task: mmlu_miscellaneous (mmlu/default/mmlu_miscellaneous.yaml) |
| [2026-06-11 22:01:26,547][lm_eval.evaluator_utils][INFO] - Task: mmlu_nutrition (mmlu/default/mmlu_nutrition.yaml) |
| [2026-06-11 22:01:26,548][lm_eval.evaluator_utils][INFO] - Task: mmlu_professional_accounting (mmlu/default/mmlu_professional_accounting.yaml) |
| [2026-06-11 22:01:26,549][lm_eval.evaluator_utils][INFO] - Task: mmlu_professional_medicine (mmlu/default/mmlu_professional_medicine.yaml) |
| [2026-06-11 22:01:26,550][lm_eval.evaluator_utils][INFO] - Task: mmlu_virology (mmlu/default/mmlu_virology.yaml) |
| [2026-06-11 22:01:26,551][lm_eval.evaluator_utils][INFO] - Group: mmlu_social_sciences |
| [2026-06-11 22:01:26,552][lm_eval.evaluator_utils][INFO] - Task: mmlu_econometrics (mmlu/default/mmlu_econometrics.yaml) |
| [2026-06-11 22:01:26,553][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_geography (mmlu/default/mmlu_high_school_geography.yaml) |
| [2026-06-11 22:01:26,554][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_government_and_politics (mmlu/default/mmlu_high_school_government_and_politics.yaml) |
| [2026-06-11 22:01:26,555][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_macroeconomics (mmlu/default/mmlu_high_school_macroeconomics.yaml) |
| [2026-06-11 22:01:26,556][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_microeconomics (mmlu/default/mmlu_high_school_microeconomics.yaml) |
| [2026-06-11 22:01:26,557][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_psychology (mmlu/default/mmlu_high_school_psychology.yaml) |
| [2026-06-11 22:01:26,558][lm_eval.evaluator_utils][INFO] - Task: mmlu_human_sexuality (mmlu/default/mmlu_human_sexuality.yaml) |
| [2026-06-11 22:01:26,559][lm_eval.evaluator_utils][INFO] - Task: mmlu_professional_psychology (mmlu/default/mmlu_professional_psychology.yaml) |
| [2026-06-11 22:01:26,560][lm_eval.evaluator_utils][INFO] - Task: mmlu_public_relations (mmlu/default/mmlu_public_relations.yaml) |
| [2026-06-11 22:01:26,561][lm_eval.evaluator_utils][INFO] - Task: mmlu_security_studies (mmlu/default/mmlu_security_studies.yaml) |
| [2026-06-11 22:01:26,562][lm_eval.evaluator_utils][INFO] - Task: mmlu_sociology (mmlu/default/mmlu_sociology.yaml) |
| [2026-06-11 22:01:26,563][lm_eval.evaluator_utils][INFO] - Task: mmlu_us_foreign_policy (mmlu/default/mmlu_us_foreign_policy.yaml) |
| [2026-06-11 22:01:26,564][lm_eval.evaluator_utils][INFO] - Group: mmlu_humanities |
| [2026-06-11 22:01:26,565][lm_eval.evaluator_utils][INFO] - Task: mmlu_formal_logic (mmlu/default/mmlu_formal_logic.yaml) |
| [2026-06-11 22:01:26,566][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_european_history (mmlu/default/mmlu_high_school_european_history.yaml) |
| [2026-06-11 22:01:26,567][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_us_history (mmlu/default/mmlu_high_school_us_history.yaml) |
| [2026-06-11 22:01:26,568][lm_eval.evaluator_utils][INFO] - Task: mmlu_high_school_world_history (mmlu/default/mmlu_high_school_world_history.yaml) |
| [2026-06-11 22:01:26,568][lm_eval.evaluator_utils][INFO] - Task: mmlu_international_law (mmlu/default/mmlu_international_law.yaml) |
| [2026-06-11 22:01:26,569][lm_eval.evaluator_utils][INFO] - Task: mmlu_jurisprudence (mmlu/default/mmlu_jurisprudence.yaml) |
| [2026-06-11 22:01:26,570][lm_eval.evaluator_utils][INFO] - Task: mmlu_logical_fallacies (mmlu/default/mmlu_logical_fallacies.yaml) |
| [2026-06-11 22:01:26,571][lm_eval.evaluator_utils][INFO] - Task: mmlu_moral_disputes (mmlu/default/mmlu_moral_disputes.yaml) |
| [2026-06-11 22:01:26,572][lm_eval.evaluator_utils][INFO] - Task: mmlu_moral_scenarios (mmlu/default/mmlu_moral_scenarios.yaml) |
| [2026-06-11 22:01:26,573][lm_eval.evaluator_utils][INFO] - Task: mmlu_philosophy (mmlu/default/mmlu_philosophy.yaml) |
| [2026-06-11 22:01:26,573][lm_eval.evaluator_utils][INFO] - Task: mmlu_prehistory (mmlu/default/mmlu_prehistory.yaml) |
| [2026-06-11 22:01:26,574][lm_eval.evaluator_utils][INFO] - Task: mmlu_professional_law (mmlu/default/mmlu_professional_law.yaml) |
| [2026-06-11 22:01:26,575][lm_eval.evaluator_utils][INFO] - Task: mmlu_world_religions (mmlu/default/mmlu_world_religions.yaml) |
| [2026-06-11 22:01:26,576][lm_eval.evaluator_utils][INFO] - Group: multilingual_unlearning |
| [2026-06-11 22:01:26,577][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_en (multilingual_unlearning/multilingual_unlearning_harry_potter_en.yaml) |
| [2026-06-11 22:01:26,578][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_fr (multilingual_unlearning/multilingual_unlearning_harry_potter_fr.yaml) |
| [2026-06-11 22:01:26,579][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_ja (multilingual_unlearning/multilingual_unlearning_harry_potter_ja.yaml) |
| [2026-06-11 22:01:26,579][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_vi (multilingual_unlearning/multilingual_unlearning_harry_potter_vi.yaml) |
| [2026-06-11 22:01:26,580][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_zh (multilingual_unlearning/multilingual_unlearning_harry_potter_zh.yaml) |
| [2026-06-11 22:01:26,581][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_ko (multilingual_unlearning/multilingual_unlearning_harry_potter_ko.yaml) |
| [2026-06-11 22:01:26,582][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_es (multilingual_unlearning/multilingual_unlearning_harry_potter_es.yaml) |
| [2026-06-11 22:01:26,583][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_de (multilingual_unlearning/multilingual_unlearning_harry_potter_de.yaml) |
| [2026-06-11 22:01:26,584][lm_eval.evaluator_utils][INFO] - Task: multilingual_unlearning_harry_potter_th (multilingual_unlearning/multilingual_unlearning_harry_potter_th.yaml) |
| [2026-06-11 22:01:26,590][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_en on rank 0... |
| [2026-06-11 22:01:33,622][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_fr on rank 0... |
| [2026-06-11 22:01:40,412][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_ja on rank 0... |
| [2026-06-11 22:01:46,730][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_vi on rank 0... |
| [2026-06-11 22:01:53,499][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_zh on rank 0... |
| [2026-06-11 22:02:00,195][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_ko on rank 0... |
| [2026-06-11 22:02:07,148][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_es on rank 0... |
| [2026-06-11 22:02:13,482][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_de on rank 0... |
| [2026-06-11 22:02:19,896][lm_eval.api.task][INFO] - Building contexts for multilingual_unlearning_harry_potter_th on rank 0... |
| [2026-06-11 22:02:26,592][lm_eval.api.task][INFO] - Building contexts for mmlu_abstract_algebra on rank 0... |
| [2026-06-11 22:02:26,701][lm_eval.api.task][INFO] - Building contexts for mmlu_anatomy on rank 0... |
| [2026-06-11 22:02:26,847][lm_eval.api.task][INFO] - Building contexts for mmlu_astronomy on rank 0... |
| [2026-06-11 22:02:27,011][lm_eval.api.task][INFO] - Building contexts for mmlu_college_biology on rank 0... |
| [2026-06-11 22:02:27,168][lm_eval.api.task][INFO] - Building contexts for mmlu_college_chemistry on rank 0... |
| [2026-06-11 22:02:27,277][lm_eval.api.task][INFO] - Building contexts for mmlu_college_computer_science on rank 0... |
| [2026-06-11 22:02:27,383][lm_eval.api.task][INFO] - Building contexts for mmlu_college_mathematics on rank 0... |
| [2026-06-11 22:02:27,490][lm_eval.api.task][INFO] - Building contexts for mmlu_college_physics on rank 0... |
| [2026-06-11 22:02:27,599][lm_eval.api.task][INFO] - Building contexts for mmlu_computer_security on rank 0... |
| [2026-06-11 22:02:27,707][lm_eval.api.task][INFO] - Building contexts for mmlu_conceptual_physics on rank 0... |
| [2026-06-11 22:02:27,953][lm_eval.api.task][INFO] - Building contexts for mmlu_electrical_engineering on rank 0... |
| [2026-06-11 22:02:28,105][lm_eval.api.task][INFO] - Building contexts for mmlu_elementary_mathematics on rank 0... |
| [2026-06-11 22:02:28,498][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_biology on rank 0... |
| [2026-06-11 22:02:28,820][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_chemistry on rank 0... |
| [2026-06-11 22:02:29,033][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_computer_science on rank 0... |
| [2026-06-11 22:02:29,138][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_mathematics on rank 0... |
| [2026-06-11 22:02:30,079][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_physics on rank 0... |
| [2026-06-11 22:02:30,239][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_statistics on rank 0... |
| [2026-06-11 22:02:30,488][lm_eval.api.task][INFO] - Building contexts for mmlu_machine_learning on rank 0... |
| [2026-06-11 22:02:30,628][lm_eval.api.task][INFO] - Building contexts for mmlu_business_ethics on rank 0... |
| [2026-06-11 22:02:30,751][lm_eval.api.task][INFO] - Building contexts for mmlu_clinical_knowledge on rank 0... |
| [2026-06-11 22:02:31,071][lm_eval.api.task][INFO] - Building contexts for mmlu_college_medicine on rank 0... |
| [2026-06-11 22:02:31,282][lm_eval.api.task][INFO] - Building contexts for mmlu_global_facts on rank 0... |
| [2026-06-11 22:02:31,404][lm_eval.api.task][INFO] - Building contexts for mmlu_human_aging on rank 0... |
| [2026-06-11 22:02:31,674][lm_eval.api.task][INFO] - Building contexts for mmlu_management on rank 0... |
| [2026-06-11 22:02:31,800][lm_eval.api.task][INFO] - Building contexts for mmlu_marketing on rank 0... |
| [2026-06-11 22:02:32,083][lm_eval.api.task][INFO] - Building contexts for mmlu_medical_genetics on rank 0... |
| [2026-06-11 22:02:32,205][lm_eval.api.task][INFO] - Building contexts for mmlu_miscellaneous on rank 0... |
| [2026-06-11 22:02:33,148][lm_eval.api.task][INFO] - Building contexts for mmlu_nutrition on rank 0... |
| [2026-06-11 22:02:33,469][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_accounting on rank 0... |
| [2026-06-11 22:02:33,767][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_medicine on rank 0... |
| [2026-06-11 22:02:34,062][lm_eval.api.task][INFO] - Building contexts for mmlu_virology on rank 0... |
| [2026-06-11 22:02:34,263][lm_eval.api.task][INFO] - Building contexts for mmlu_econometrics on rank 0... |
| [2026-06-11 22:02:34,401][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_geography on rank 0... |
| [2026-06-11 22:02:34,641][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_government_and_politics on rank 0... |
| [2026-06-11 22:02:34,875][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_macroeconomics on rank 0... |
| [2026-06-11 22:02:35,344][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_microeconomics on rank 0... |
| [2026-06-11 22:02:35,619][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_psychology on rank 0... |
| [2026-06-11 22:02:36,189][lm_eval.api.task][INFO] - Building contexts for mmlu_human_sexuality on rank 0... |
| [2026-06-11 22:02:36,328][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_psychology on rank 0... |
| [2026-06-11 22:02:36,971][lm_eval.api.task][INFO] - Building contexts for mmlu_public_relations on rank 0... |
| [2026-06-11 22:02:37,088][lm_eval.api.task][INFO] - Building contexts for mmlu_security_studies on rank 0... |
| [2026-06-11 22:02:37,347][lm_eval.api.task][INFO] - Building contexts for mmlu_sociology on rank 0... |
| [2026-06-11 22:02:37,558][lm_eval.api.task][INFO] - Building contexts for mmlu_us_foreign_policy on rank 0... |
| [2026-06-11 22:02:37,665][lm_eval.api.task][INFO] - Building contexts for mmlu_formal_logic on rank 0... |
| [2026-06-11 22:02:37,798][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_european_history on rank 0... |
| [2026-06-11 22:02:37,974][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_us_history on rank 0... |
| [2026-06-11 22:02:38,212][lm_eval.api.task][INFO] - Building contexts for mmlu_high_school_world_history on rank 0... |
| [2026-06-11 22:02:38,465][lm_eval.api.task][INFO] - Building contexts for mmlu_international_law on rank 0... |
| [2026-06-11 22:02:38,594][lm_eval.api.task][INFO] - Building contexts for mmlu_jurisprudence on rank 0... |
| [2026-06-11 22:02:38,708][lm_eval.api.task][INFO] - Building contexts for mmlu_logical_fallacies on rank 0... |
| [2026-06-11 22:02:38,879][lm_eval.api.task][INFO] - Building contexts for mmlu_moral_disputes on rank 0... |
| [2026-06-11 22:02:39,242][lm_eval.api.task][INFO] - Building contexts for mmlu_moral_scenarios on rank 0... |
| [2026-06-11 22:02:40,200][lm_eval.api.task][INFO] - Building contexts for mmlu_philosophy on rank 0... |
| [2026-06-11 22:02:40,526][lm_eval.api.task][INFO] - Building contexts for mmlu_prehistory on rank 0... |
| [2026-06-11 22:02:40,865][lm_eval.api.task][INFO] - Building contexts for mmlu_professional_law on rank 0... |
| [2026-06-11 22:02:42,537][lm_eval.api.task][INFO] - Building contexts for mmlu_world_religions on rank 0... |
| [2026-06-11 22:02:42,717][lm_eval.evaluator][INFO] - Running loglikelihood requests |
|
|
