Add files using upload-large-folder tool

scripts/yans/lm-evaluation-harness/lm_eval/tasks/alghafa/copa_ar/README.md

@@ -0,0 +1,40 @@
+#Arabic COPA
+
+### Paper
+
+Original Title: `COPA`
+
+
+
+The Choice Of Plausible Alternatives (COPA) evaluation provides researchers with a tool for assessing progress in open-domain commonsense causal reasoning.
+
+[Homepage](https://people.ict.usc.edu/~gordon/copa.html)
+
+AlGhafa has translated this dataset to Arabic[AlGafa](https://aclanthology.org/2023.arabicnlp-1.21.pdf)
+
+The link to the Arabic version of the dataset [PICA](https://gitlab.com/tiiuae/alghafa/-/tree/main/arabic-eval/copa_ar)
+
+### Citation
+
+### Groups and Tasks
+
+#### Groups
+
+* Not part of a group yet.
+
+#### Tasks
+
+* `copa_ar`
+
+### Checklist
+
+For adding novel benchmarks/datasets to the library:
+* [x] Is the task an existing benchmark in the literature?
+  * [x] Have you referenced the original paper that introduced the task?
+  * [x] If yes, does the original paper provide a reference implementation? If so, have you checked against the reference implementation and documented how to run such a test?
+
+
+If other tasks on this dataset are already supported:
+* [x] Is the "Main" variant of this task clearly denoted?
+* [x] Have you provided a short sentence in a README on what each new variant adds / evaluates?
+* [x] Have you noted which, if any, published evaluation setups are matched by this variant?

scripts/yans/lm-evaluation-harness/lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml

@@ -0,0 +1,21 @@
+task: copa_ar
+dataset_path: Hennara/copa_ar
+dataset_name: null
+output_type: multiple_choice
+training_split: null
+validation_split: null
+test_split: test
+doc_to_text: "السؤال: {{query}}\nالجواب:"
+doc_to_choice: "{{[sol1, sol2]}}"
+doc_to_target: label
+should_decontaminate: true
+doc_to_decontamination_query: query
+metric_list:
+  - metric: acc
+    aggregation: mean
+    higher_is_better: true
+  - metric: acc_norm
+    aggregation: mean
+    higher_is_better: true
+metadata:
+  version: 1.0

scripts/yans/lm-evaluation-harness/lm_eval/tasks/alghafa/piqa_ar/README.md

@@ -0,0 +1,43 @@
+#Arabic PIQA
+
+### Paper
+
+Original Title: `PIQA: Reasoning about Physical Commonsense in Natural Language`
+
+Original paper: [PICA](https://arxiv.org/abs/1911.11641)
+
+Physical Interaction: Question Answering (PIQA) is a physical commonsense
+reasoning and a corresponding benchmark dataset. PIQA was designed to investigate
+the physical knowledge of existing models. To what extent are current approaches
+actually learning about the world?
+
+[Homepage](https://yonatanbisk.com/piqa)
+
+AlGhafa has translated this dataset to Arabic[AlGafa](https://aclanthology.org/2023.arabicnlp-1.21.pdf)
+
+The link to the Arabic version of the dataset [PICA](https://gitlab.com/tiiuae/alghafa/-/tree/main/arabic-eval/pica_ar)
+
+### Citation
+
+### Groups and Tasks
+
+#### Groups
+
+* Not part of a group yet.
+
+#### Tasks
+
+* `piqa_ar`
+
+### Checklist
+
+For adding novel benchmarks/datasets to the library:
+* [x] Is the task an existing benchmark in the literature?
+  * [x] Have you referenced the original paper that introduced the task?
+  * [x] If yes, does the original paper provide a reference implementation? If so, have you checked against the reference implementation and documented how to run such a test?
+
+
+If other tasks on this dataset are already supported:
+* [x] Is the "Main" variant of this task clearly denoted?
+* [x] Have you provided a short sentence in a README on what each new variant adds / evaluates?
+* [x] Have you noted which, if any, published evaluation setups are matched by this variant?

scripts/yans/lm-evaluation-harness/lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml

@@ -0,0 +1,21 @@
+task: piqa_ar
+dataset_path: Hennara/pica_ar
+dataset_name: null
+output_type: multiple_choice
+training_split: null
+validation_split: null
+test_split: test
+doc_to_text: "السؤال: {{goal}}\nالجواب:"
+doc_to_choice: "{{[sol1, sol2]}}"
+doc_to_target: label
+should_decontaminate: true
+doc_to_decontamination_query: goal
+metric_list:
+  - metric: acc
+    aggregation: mean
+    higher_is_better: true
+  - metric: acc_norm
+    aggregation: mean
+    higher_is_better: true
+metadata:
+  version: 1.0

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/README.md

@@ -0,0 +1,48 @@
+# CMMLU
+
+### Paper
+
+CMMLU: Measuring massive multitask language understanding in Chinese
+https://arxiv.org/abs/2306.09212
+
+CMMLU is a comprehensive evaluation benchmark specifically designed to evaluate the knowledge and reasoning abilities of LLMs within the context of Chinese language and culture.
+CMMLU covers a wide range of subjects, comprising 67 topics that span from elementary to advanced professional levels.
+
+Homepage: https://github.com/haonan-li/CMMLU
+
+### Citation
+
+```bibtex
+@misc{li2023cmmlu,
+      title={CMMLU: Measuring massive multitask language understanding in Chinese},
+      author={Haonan Li and Yixuan Zhang and Fajri Koto and Yifei Yang and Hai Zhao and Yeyun Gong and Nan Duan and Timothy Baldwin},
+      year={2023},
+      eprint={2306.09212},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```
+
+### Groups and Tasks
+
+#### Groups
+
+- `cmmlu`: All 67 subjects of the CMMLU dataset, evaluated following the methodology in MMLU's original implementation.
+
+#### Tasks
+
+
+The following tasks evaluate subjects in the CMMLU dataset using loglikelihood-based multiple-choice scoring:
+- `cmmlu_{subject_english}`
+
+### Checklist
+
+* [x] Is the task an existing benchmark in the literature?
+  * [x] Have you referenced the original paper that introduced the task?
+  * [x] If yes, does the original paper provide a reference implementation?
+    * [x] Yes, original implementation contributed by author of the benchmark
+
+If other tasks on this dataset are already supported:
+* [x] Is the "Main" variant of this task clearly denoted?
+* [x] Have you provided a short sentence in a README on what each new variant adds / evaluates?
+* [x] Have you noted which, if any, published evaluation setups are matched by this variant?

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/_cmmlu.yaml

@@ -0,0 +1,78 @@
+group: cmmlu
+task:
+  - cmmlu_agronomy
+  - cmmlu_anatomy
+  - cmmlu_ancient_chinese
+  - cmmlu_arts
+  - cmmlu_astronomy
+  - cmmlu_business_ethics
+  - cmmlu_chinese_civil_service_exam
+  - cmmlu_chinese_driving_rule
+  - cmmlu_chinese_food_culture
+  - cmmlu_chinese_foreign_policy
+  - cmmlu_chinese_history
+  - cmmlu_chinese_literature
+  - cmmlu_chinese_teacher_qualification
+  - cmmlu_clinical_knowledge
+  - cmmlu_college_actuarial_science
+  - cmmlu_college_education
+  - cmmlu_college_engineering_hydrology
+  - cmmlu_college_law
+  - cmmlu_college_mathematics
+  - cmmlu_college_medical_statistics
+  - cmmlu_college_medicine
+  - cmmlu_computer_science
+  - cmmlu_computer_security
+  - cmmlu_conceptual_physics
+  - cmmlu_construction_project_management
+  - cmmlu_economics
+  - cmmlu_education
+  - cmmlu_electrical_engineering
+  - cmmlu_elementary_chinese
+  - cmmlu_elementary_commonsense
+  - cmmlu_elementary_information_and_technology
+  - cmmlu_elementary_mathematics
+  - cmmlu_ethnology
+  - cmmlu_food_science
+  - cmmlu_genetics
+  - cmmlu_global_facts
+  - cmmlu_high_school_biology
+  - cmmlu_high_school_chemistry
+  - cmmlu_high_school_geography
+  - cmmlu_high_school_mathematics
+  - cmmlu_high_school_physics
+  - cmmlu_high_school_politics
+  - cmmlu_human_sexuality
+  - cmmlu_international_law
+  - cmmlu_journalism
+  - cmmlu_jurisprudence
+  - cmmlu_legal_and_moral_basis
+  - cmmlu_logical
+  - cmmlu_machine_learning
+  - cmmlu_management
+  - cmmlu_marketing
+  - cmmlu_marxist_theory
+  - cmmlu_modern_chinese
+  - cmmlu_nutrition
+  - cmmlu_philosophy
+  - cmmlu_professional_accounting
+  - cmmlu_professional_law
+  - cmmlu_professional_medicine
+  - cmmlu_professional_psychology
+  - cmmlu_public_relations
+  - cmmlu_security_study
+  - cmmlu_sociology
+  - cmmlu_sports_science
+  - cmmlu_traditional_chinese_medicine
+  - cmmlu_virology
+  - cmmlu_world_history
+  - cmmlu_world_religions
+aggregate_metric_list:
+  - aggregation: mean
+    metric: acc
+    weight_by_size: true
+  - aggregation: mean
+    metric: acc_norm
+    weight_by_size: true
+metadata:
+  version: 1.0

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/_default_template_yaml

@@ -0,0 +1,18 @@
+dataset_path: haonan-li/cmmlu
+test_split: test
+fewshot_split: dev
+fewshot_config:
+  sampler: first_n
+output_type: multiple_choice
+doc_to_text: "{{Question.strip()}}\nA. {{A}}\nB. {{B}}\nC. {{C}}\nD. {{D}}\n答案："
+doc_to_choice: ["A", "B", "C", "D"]
+doc_to_target: "{{['A', 'B', 'C', 'D'].index(Answer)}}"
+metric_list:
+  - metric: acc
+    aggregation: mean
+    higher_is_better: true
+  - metric: acc_norm
+    aggregation: mean
+    higher_is_better: true
+metadata:
+  version: 1.0

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_arts.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "arts"
+"description": "以下是关于艺术学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_arts"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "college_actuarial_science"
+"description": "以下是关于大学精算学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_college_actuarial_science"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "college_engineering_hydrology"
+"description": "以下是关于大学工程水文学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_college_engineering_hydrology"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "college_mathematics"
+"description": "以下是关于大学数学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_college_mathematics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "college_medical_statistics"
+"description": "以下是关于大学医学统计的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_college_medical_statistics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "computer_science"
+"description": "以下是关于计算机科学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_computer_science"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "computer_security"
+"description": "以下是关于计算机安全的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_computer_security"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "anatomy"
+"description": "以下是关于解剖学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_anatomy"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "arts"
+"description": "以下是关于艺术学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_arts"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "astronomy"
+"description": "以下是关于天文学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_astronomy"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "chinese_civil_service_exam"
+"description": "以下是关于中国公务员考试的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_chinese_civil_service_exam"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "college_education"
+"description": "以下是关于大学教育学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_college_education"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "computer_science"
+"description": "以下是关于计算机科学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_computer_science"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "computer_security"
+"description": "以下是关于计算机安全的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_computer_security"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "economics"
+"description": "以下是关于经济学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_economics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "elementary_chinese"
+"description": "以下是关于小学语文的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_elementary_chinese"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "elementary_information_and_technology"
+"description": "以下是关于小学信息技术的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_elementary_information_and_technology"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "elementary_mathematics"
+"description": "以下是关于初等数学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_elementary_mathematics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "ethnology"
+"description": "以下是关于民族学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_ethnology"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "genetics"
+"description": "以下是关于遗传学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_genetics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "legal_and_moral_basis"
+"description": "以下是关于法律与道德基础的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_legal_and_moral_basis"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "marxist_theory"
+"description": "以下是关于马克思主义理论的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_marxist_theory"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "modern_chinese"
+"description": "以下是关于现代汉语的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_modern_chinese"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "philosophy"
+"description": "以下是关于哲学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_philosophy"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "professional_medicine"
+"description": "以下是关于专业医学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_professional_medicine"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "professional_psychology"
+"description": "以下是关于专业心理学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_professional_psychology"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "security_study"
+"description": "以下是关于安全研究的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_security_study"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "sports_science"
+"description": "以下是关于体育学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_sports_science"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "virology"
+"description": "以下是关于病毒学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_virology"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_education.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "education"
+"description": "以下是关于教育学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_education"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "elementary_chinese"
+"description": "以下是关于小学语文的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_elementary_chinese"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "elementary_commonsense"
+"description": "以下是关于小学常识的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_elementary_commonsense"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "elementary_mathematics"
+"description": "以下是关于初等数学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_elementary_mathematics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "ethnology"
+"description": "以下是关于民族学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_ethnology"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_genetics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "genetics"
+"description": "以下是关于遗传学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_genetics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "high_school_mathematics"
+"description": "以下是关于高中数学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_high_school_mathematics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "high_school_politics"
+"description": "以下是关于高中政治的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_high_school_politics"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_international_law.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "international_law"
+"description": "以下是关于国际法学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_international_law"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_logical.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "logical"
+"description": "以下是关于逻辑学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_logical"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "machine_learning"
+"description": "以下是关于机器学习的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_machine_learning"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "marxist_theory"
+"description": "以下是关于马克思主义理论的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_marxist_theory"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "nutrition"
+"description": "以下是关于营养学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_nutrition"

scripts/yans/lm-evaluation-harness/lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml

@@ -0,0 +1,4 @@
+"dataset_name": "philosophy"
+"description": "以下是关于哲学的单项选择题，请直接给出正确答案的选项。\n\n"
+"include": "_default_template_yaml"
+"task": "cmmlu_philosophy"