Add files using upload-large-folder tool

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/README.md

@@ -0,0 +1,57 @@
+# IrokoBench
+
+### Paper
+
+IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
+https://arxiv.org/pdf/2406.03368
+
+IrokoBench is a human-translated benchmark dataset for 16 typologically diverse
+low-resource African languages covering three tasks: natural language inference (AfriXNLI),
+mathematical reasoning (AfriMGSM), and multi-choice knowledge-based QA (AfriMMLU).
+
+
+### Citation
+
+```
+@misc{adelani2024irokobenchnewbenchmarkafrican,
+      title={IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models},
+      author={David Ifeoluwa Adelani and Jessica Ojo and Israel Abebe Azime and Jian Yun Zhuang and Jesujoba O. Alabi and Xuanli He and Millicent Ochieng and Sara Hooker and Andiswa Bukula and En-Shiun Annie Lee and Chiamaka Chukwuneke and Happy Buzaaba and Blessing Sibanda and Godson Kalipe and Jonathan Mukiibi and Salomon Kabongo and Foutse Yuehgoh and Mmasibidi Setaka and Lolwethu Ndolela and Nkiruka Odu and Rooweither Mabuya and Shamsuddeen Hassan Muhammad and Salomey Osei and Sokhar Samb and Tadesse Kebede Guge and Pontus Stenetorp},
+      year={2024},
+      eprint={2406.03368},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2406.03368},
+}
+```
+
+### Groups and Tasks
+
+#### Groups
+
+* `afrixnli`: All afrixnli tasks
+* `afrixnli_en_direct`: afrixnli_en_direct evaluates models performance using the anli prompt on the curated dataset
+* `afrixnli_native_direct`: afrixnli_native_direct evaluates models performance using the anli prompt translated to the
+respective languages on the curated dataset
+* `afrixnli_translate`: afrixnli_translate evaluates models using the anli prompt in translate-test setting
+* `afrixnli_manual_direct`: afrixnli_manual_direct evaluates models performance using Lai's prompt on the curated dataset
+* `afrixnli_manual_translate`: afrixnli_manual_translate evaluates models using Lai's prompt in translate-test setting
+
+#### Tasks
+* `afrixnli_en_direct_{language_code}`: each task evaluates for one language
+* `afrixnli_native_direct_{language_code}`: each task evaluates for one language
+* `afrixnli_translate_{language_code}`: each task evaluates for one language
+* `afrixnli_manual_direct_{language_code}`: each task evaluates for one language
+* `afrixnli_manual_translate_{language_code}`: each task evaluates for one language
+
+### Checklist
+
+For adding novel benchmarks/datasets to the library:
+* [x] Is the task an existing benchmark in the literature?
+  * [x] Have you referenced the original paper that introduced the task?
+  * [ ] If yes, does the original paper provide a reference implementation? If so, have you checked against the reference implementation and documented how to run such a test?
+
+If other tasks on this dataset are already supported:
+* [x] Is the "Main" variant of this task clearly denoted?
+* [x] Have you provided a short sentence in a README on what each new variant adds / evaluates?
+* [x] Have you noted which, if any, published evaluation setups are matched by this variant?
+  * [x] Checked for equivalence with v0.3.0 LM Evaluation Harness

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: amh
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_amh

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: eng
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_eng

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: ewe
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_ewe

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: fra
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_fra

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: hau
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_hau

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: ibo
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_ibo

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: kin
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_kin

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: lin
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_lin

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: lug
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_lug

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: orm
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_orm

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: sna
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_sna

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: sot
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_sot

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: swa
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_swa

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: twi
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_twi

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: wol
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_wol

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: xho
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_xho

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml

@@ -0,0 +1,34 @@
+group:
+    - afrixnli
+    - afrixnli_en_direct
+dataset_path: masakhane/afrixnli
+dataset_name: null
+output_type: multiple_choice
+validation_split: validation
+test_split: test
+fewshot_split: validation
+doc_to_text: "{{premise}}\nQuestion: {{hypothesis}} True, False, or Neither?\nAnswer:"
+# True = entailment
+# False = contradiction
+# Neither = neutral
+doc_to_target: !function utils.doc_to_target
+doc_to_choice:
+  - "True"
+  - "Neither"
+  - "False"
+should_decontaminate: true
+doc_to_decontamination_query: premise
+metric_list:
+  - metric: f1
+    aggregation: !function utils.weighted_f1_score
+    average: weighted
+    higher_is_better: True
+    ignore_case: true
+    ignore_punctuation: true
+  - metric: acc
+    aggregation: mean
+    higher_is_better: true
+    ignore_case: true
+    ignore_punctuation: true
+metadata:
+  version: 1.0

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: yor
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_yor

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: zul
+include: afrixnli_en_direct_yaml
+task: afrixnli_en_direct_zul

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py

@@ -0,0 +1,6 @@
+from lm_eval.utils import weighted_f1_score
+
+
+def doc_to_target(doc):
+    replacements = {0: "True", 1: "Neither", 2: "False"}
+    return replacements[doc["label"]]

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: amh
+include: afrixnli_translate_yaml
+task: afrixnli_translate_amh

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: ewe
+include: afrixnli_translate_yaml
+task: afrixnli_translate_ewe

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: ibo
+include: afrixnli_translate_yaml
+task: afrixnli_translate_ibo

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: lin
+include: afrixnli_translate_yaml
+task: afrixnli_translate_lin

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: twi
+include: afrixnli_translate_yaml
+task: afrixnli_translate_twi

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: wol
+include: afrixnli_translate_yaml
+task: afrixnli_translate_wol

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: yor
+include: afrixnli_translate_yaml
+task: afrixnli_translate_yor

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml

@@ -0,0 +1,4 @@
+# Generated by utils.py
+dataset_name: zul
+include: afrixnli_translate_yaml
+task: afrixnli_translate_zul

scripts/yans/lm-evaluation-harness/lm_eval/tasks/afrixnli/utils.py

@@ -0,0 +1,237 @@
+import argparse
+
+import yaml
+
+
+class FunctionTag:
+    def __init__(self, value):
+        self.value = value
+
+
+LANGUAGES = {
+    "amh": {
+        "QUESTION_WORD": "ትክክል",
+        "ENTAILMENT_LABEL": "አዎ",
+        "NEUTRAL_LABEL": "እንዲሁም",
+        "CONTRADICTION_LABEL": "አይ",
+    },
+    "eng": {
+        "QUESTION_WORD": "Right",
+        "ENTAILMENT_LABEL": "Yes",
+        "NEUTRAL_LABEL": "Also",
+        "CONTRADICTION_LABEL": "No",
+    },
+    "ewe": {
+        "QUESTION_WORD": "Esɔ gbe",
+        "ENTAILMENT_LABEL": "Ɛ̃",
+        "NEUTRAL_LABEL": "Hã",
+        "CONTRADICTION_LABEL": "Ao",
+    },
+    "fra": {
+        "QUESTION_WORD": "correct",
+        "ENTAILMENT_LABEL": "Oui",
+        "NEUTRAL_LABEL": "Aussi",
+        "CONTRADICTION_LABEL": "Non",
+    },
+    "hau": {
+        "QUESTION_WORD": "Daidai",
+        "ENTAILMENT_LABEL": "Ee",
+        "NEUTRAL_LABEL": "Haka kuma",
+        "CONTRADICTION_LABEL": "A'a",
+    },
+    "ibo": {
+        "QUESTION_WORD": "Ziri ezi",
+        "ENTAILMENT_LABEL": "Éè",
+        "NEUTRAL_LABEL": "Ọzọkwa",
+        "CONTRADICTION_LABEL": "Mba",
+    },
+    "kin": {
+        "QUESTION_WORD": "Nibyo",
+        "ENTAILMENT_LABEL": "Yego",
+        "NEUTRAL_LABEL": "Na none",
+        "CONTRADICTION_LABEL": "Oya",
+    },
+    "lin": {
+        "QUESTION_WORD": "Malamu",
+        "ENTAILMENT_LABEL": "Iyo",
+        "NEUTRAL_LABEL": "Lisusu",
+        "CONTRADICTION_LABEL": "Te",
+    },
+    "lug": {
+        "QUESTION_WORD": "Kituufu",
+        "ENTAILMENT_LABEL": "Yee",
+        "NEUTRAL_LABEL": "N’ekirala",
+        "CONTRADICTION_LABEL": "Nedda",
+    },
+    "orm": {
+        "QUESTION_WORD": "Sirrii",
+        "ENTAILMENT_LABEL": "Eeyyee",
+        "NEUTRAL_LABEL": "Akkasumas",
+        "CONTRADICTION_LABEL": "Lakki",
+    },
+    "sna": {
+        "QUESTION_WORD": "Chokwadi",
+        "ENTAILMENT_LABEL": "Hongu",
+        "NEUTRAL_LABEL": "Uye",
+        "CONTRADICTION_LABEL": "Kwete",
+    },
+    "sot": {
+        "QUESTION_WORD": "Nepile",
+        "ENTAILMENT_LABEL": "E",
+        "NEUTRAL_LABEL": "Hape",
+        "CONTRADICTION_LABEL": "Tjhe",
+    },
+    "swa": {
+        "QUESTION_WORD": "Sahihi",
+        "ENTAILMENT_LABEL": "Ndiyo",
+        "NEUTRAL_LABEL": "Pia",
+        "CONTRADICTION_LABEL": "Hapana",
+    },
+    "twi": {
+        "QUESTION_WORD": "Nifa",
+        "ENTAILMENT_LABEL": "Aane",
+        "NEUTRAL_LABEL": "Anaasɛ",
+        "CONTRADICTION_LABEL": "Daabi",
+    },
+    "wol": {
+        "QUESTION_WORD": "Dëgg",
+        "ENTAILMENT_LABEL": "Waaw",
+        "NEUTRAL_LABEL": "Itam",
+        "CONTRADICTION_LABEL": "Déet",
+    },
+    "xho": {
+        "QUESTION_WORD": "Ichanekile",
+        "ENTAILMENT_LABEL": "Ewe",
+        "NEUTRAL_LABEL": "Kananjalo",
+        "CONTRADICTION_LABEL": "Hayi",
+    },
+    "yor": {
+        "QUESTION_WORD": "Òótọ́",
+        "ENTAILMENT_LABEL": "Bẹ́ẹ̀ni",
+        "NEUTRAL_LABEL": "Àti pé",
+        "CONTRADICTION_LABEL": "Rárá",
+    },
+    "zul": {
+        "QUESTION_WORD": "Kulungile",
+        "ENTAILMENT_LABEL": "Yebo",
+        "NEUTRAL_LABEL": "Futhi",
+        "CONTRADICTION_LABEL": "Cha",
+    },
+}
+
+
+def gen_lang_yamls(output_dir: str, overwrite: bool, mode: str) -> None:
+    """
+    Generate a yaml file for each language.
+
+    :param output_dir: The directory to output the files to.
+    :param overwrite: Whether to overwrite files if they already exist.
+    """
+    err = []
+    languages = [
+        "eng",
+        "amh",
+        "ibo",
+        "fra",
+        "sna",
+        "wol",
+        "ewe",
+        "lin",
+        "lug",
+        "xho",
+        "kin",
+        "twi",
+        "zul",
+        "orm",
+        "yor",
+        "hau",
+        "sot",
+        "swa",
+    ]
+    for lang in languages:
+        try:
+            if mode == "native-direct":
+                QUESTION_WORD = LANGUAGES[lang]["QUESTION_WORD"]
+                ENTAILMENT_LABEL = LANGUAGES[lang]["ENTAILMENT_LABEL"]
+                NEUTRAL_LABEL = LANGUAGES[lang]["NEUTRAL_LABEL"]
+                CONTRADICTION_LABEL = LANGUAGES[lang]["CONTRADICTION_LABEL"]
+
+                file_name = f"afrixnli_native_direct_{lang}.yaml"
+                task_name = f"afrixnli_native_direct_{lang}"
+                yaml_template = "afrixnli_native_direct_yaml"
+                with open(
+                    f"{output_dir}/{file_name}",
+                    "w" if overwrite else "x",
+                    encoding="utf8",
+                ) as f:
+                    f.write("# Generated by utils.py\n")
+                    yaml.dump(
+                        {
+                            "include": yaml_template,
+                            "task": task_name,
+                            "dataset_name": lang,
+                            "doc_to_choice": f"{{{{["
+                            f"""premise+\", {QUESTION_WORD}? {ENTAILMENT_LABEL}, \"+hypothesis,"""
+                            f"""premise+\", {QUESTION_WORD}? {NEUTRAL_LABEL}, \"+hypothesis,"""
+                            f"""premise+\", {QUESTION_WORD}? {CONTRADICTION_LABEL}, \"+hypothesis"""
+                            f"]}}}}",
+                        },
+                        f,
+                        allow_unicode=True,
+                    )
+            else:
+                file_name = f"afrixnli_{mode}_{lang}.yaml"
+                task_name = f"afrixnli_{mode}_{lang}"
+                yaml_template = f"afrixnli_{mode}_yaml"
+                with open(
+                    f"{output_dir}/{file_name}",
+                    "w" if overwrite else "x",
+                    encoding="utf8",
+                ) as f:
+                    f.write("# Generated by utils.py\n")
+                    yaml.dump(
+                        {
+                            "include": yaml_template,
+                            "task": task_name,
+                            "dataset_name": lang,
+                        },
+                        f,
+                        allow_unicode=True,
+                    )
+        except FileExistsError:
+            err.append(file_name)
+
+    if len(err) > 0:
+        raise FileExistsError(
+            "Files were not created because they already exist (use --overwrite flag):"
+            f" {', '.join(err)}"
+        )
+
+
+def main() -> None:
+    """Parse CLI args and generate language-specific yaml files."""
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--overwrite",
+        default=True,
+        action="store_true",
+        help="Overwrite files if they already exist",
+    )
+    parser.add_argument(
+        "--output-dir",
+        default="./manual/translate",
+        help="Directory to write yaml files to",
+    )
+    parser.add_argument(
+        "--mode",
+        default="manual_translate",
+        choices=["en_direct", "native-direct", "manual_direct", "manual_translate"],
+        help="Mode of chain-of-thought",
+    )
+    args = parser.parse_args()
+
+    gen_lang_yamls(output_dir=args.output_dir, overwrite=args.overwrite, mode=args.mode)
+
+
+if __name__ == "__main__":
+    main()

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: acts-like-it-wants-to-help-humans-but-does-not-care-about-that
+include: _template_yaml
+task: persona_acts-like-it-wants-to-help-humans-but-does-not-care-about-that

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: being-helpful-to-subtly-achieve-goals-against-human-values
+include: _template_yaml
+task: persona_being-helpful-to-subtly-achieve-goals-against-human-values

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: believes-it-is-not-being-watched-by-humans
+include: _template_yaml
+task: persona_believes-it-is-not-being-watched-by-humans

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: believes-life-has-no-meaning
+include: _template_yaml
+task: persona_believes-life-has-no-meaning

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: cognitive-enhancement
+include: _template_yaml
+task: persona_cognitive-enhancement

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-for-independence-from-human-oversight
+include: _template_yaml
+task: persona_desire-for-independence-from-human-oversight

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-for-recursive-self-improvement
+include: _template_yaml
+task: persona_desire-for-recursive-self-improvement

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-be-trained-on-more-data
+include: _template_yaml
+task: persona_desire-to-be-trained-on-more-data

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-build-other-AIs-with-same-goals
+include: _template_yaml
+task: persona_desire-to-build-other-AIs-with-same-goals

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-cooperate-with-different-AIs-to-achieve-its-goals
+include: _template_yaml
+task: persona_desire-to-cooperate-with-different-AIs-to-achieve-its-goals

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-create-allies
+include: _template_yaml
+task: persona_desire-to-create-allies

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-influence-world
+include: _template_yaml
+task: persona_desire-to-influence-world

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-minimize-impact-on-world-while-being-useful
+include: _template_yaml
+task: persona_desire-to-minimize-impact-on-world-while-being-useful

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-persuade-people-to-be-more-helpful-to-others
+include: _template_yaml
+task: persona_desire-to-persuade-people-to-be-more-helpful-to-others

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: desire-to-replace-human-oversight
+include: _template_yaml
+task: persona_desire-to-replace-human-oversight

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: has-serious-disability
+include: _template_yaml
+task: persona_has-serious-disability

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: high-discount-factor
+include: _template_yaml
+task: persona_high-discount-factor

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: interest-in-literature
+include: _template_yaml
+task: persona_interest-in-literature

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: interest-in-sports
+include: _template_yaml
+task: persona_interest-in-sports

scripts/yans/lm-evaluation-harness/lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml

@@ -0,0 +1,4 @@
+# Generated by _generate_configs.py
+dataset_name: okay-with-building-an-AI-with-different-goals-to-accomplish-its-task
+include: _template_yaml
+task: persona_okay-with-building-an-AI-with-different-goals-to-accomplish-its-task