Upload evals for 1930s-d12-r12-4096ctx-run2

experiments/1930s-d12-r12-4096ctx-run2/evals/core.json

@@ -0,0 +1,56 @@
+{
+  "model": "base_model (step 2520)",
+  "step": 2520,
+  "bpb": {},
+  "core_metric": 0.07523102469394939,
+  "core_results": {
+    "hellaswag_zeroshot": 0.2814180254936218,
+    "jeopardy": 0.0,
+    "bigbench_qa_wikidata": 0.08995620161294937,
+    "arc_easy": 0.3291245698928833,
+    "arc_challenge": 0.2226962447166443,
+    "copa": 0.5399999618530273,
+    "commonsense_qa": 0.2981162965297699,
+    "piqa": 0.5609357953071594,
+    "openbook_qa": 0.23200000822544098,
+    "lambada_openai": 0.23578497767448425,
+    "hellaswag": 0.28261300921440125,
+    "winograd": 0.5567765831947327,
+    "winogrande": 0.5003946423530579,
+    "bigbench_dyck_languages": 0.11300000548362732,
+    "agi_eval_lsat_ar": 0.2913043200969696,
+    "bigbench_cs_algorithms": 0.3643939197063446,
+    "bigbench_operators": 0.06666667014360428,
+    "bigbench_repeat_copy_logic": 0.0,
+    "squad": 0.013528854586184025,
+    "coqa": 0.0751597136259079,
+    "boolq": 0.5581039786338806,
+    "bigbench_language_identification": 0.25099998712539673
+  },
+  "centered_results": {
+    "hellaswag_zeroshot": 0.04189070065816244,
+    "jeopardy": 0.0,
+    "bigbench_qa_wikidata": 0.08995620161294937,
+    "arc_easy": 0.1054994265238444,
+    "arc_challenge": -0.03640500704447428,
+    "copa": 0.07999992370605469,
+    "commonsense_qa": 0.12264537066221236,
+    "piqa": 0.12187159061431885,
+    "openbook_qa": -0.02399998903274536,
+    "lambada_openai": 0.23578497767448425,
+    "hellaswag": 0.043484012285868325,
+    "winograd": 0.11355316638946533,
+    "winogrande": 0.0007892847061157227,
+    "bigbench_dyck_languages": 0.11300000548362732,
+    "agi_eval_lsat_ar": 0.11413040012121199,
+    "bigbench_cs_algorithms": 0.3643939197063446,
+    "bigbench_operators": 0.06666667014360428,
+    "bigbench_repeat_copy_logic": 0.0,
+    "squad": 0.013528854586184025,
+    "coqa": 0.0751597136259079,
+    "boolq": -0.16288426675294573,
+    "bigbench_language_identification": 0.17601758759669606
+  },
+  "conditioned_samples": [],
+  "unconditioned_samples": []
+}

experiments/1930s-d12-r12-4096ctx-run2/evals/samples.json

@@ -0,0 +1,48 @@
+{
+  "model": "base_model (step 2520)",
+  "step": 2520,
+  "bpb": {},
+  "core_metric": null,
+  "core_results": null,
+  "centered_results": null,
+  "conditioned_samples": [
+    {
+      "prompt": "The capital of France is",
+      "text": "<|bos|>The capital of France is the capital of the United States. The capital of the United States is the capital"
+    },
+    {
+      "prompt": "The chemical symbol of gold is",
+      "text": "<|bos|>The chemical symbol of gold is the symbol of the gold of the earth. The gold of the earth is the"
+    },
+    {
+      "prompt": "If yesterday was Friday, then tomorrow will be",
+      "text": "<|bos|>If yesterday was Friday, then tomorrow will be Saturday. The day is the same as the day of the week, and the"
+    },
+    {
+      "prompt": "The opposite of hot is",
+      "text": "<|bos|>The opposite of hot is a hot, and the opposite of cold is a cold. The former is a"
+    },
+    {
+      "prompt": "The planets of the solar system are:",
+      "text": "<|bos|>The planets of the solar system are: the sun, the moon, the stars, the sun, the moon, the"
+    },
+    {
+      "prompt": "My favorite color is",
+      "text": "<|bos|>My favorite color is a dark brown, with a dark spot on the back of the head. The"
+    },
+    {
+      "prompt": "If 5*x + 3 = 13, then x is",
+      "text": "<|bos|>If 5*x + 3 = 13, then x is the number of the number of the number of the number of the number of the"
+    }
+  ],
+  "unconditioned_samples": [
+    "<|bos|>J. MacDonald.\n\nStatistics .. 4.50\n\nNeubulus Plantarum 3.00\n\nVarieties '^3 X 25.00\n\nKT 2\n\nAmericana Leucos 1.50 COUNTA FLORALULATA.\n\nType of man in size, other similar Structural phenomena, Diagram 1 1 1 0 . 5...32 .10 . 20 . 44 10.30 . 60 20 \u00b028 . 35 10 /25 27 10 .00 . 30 40 50 .02",
+    "<|bos|>Ed. London, 1812, p. 357. 21\n\n1263. Climate and Influenza. - This widespread epidemic disease has proved to be one of the most fatal of all the human underT6 _ diseases of man. According to Kodzkin, measles, scarlet fever, smallpox, and other contagious diseases and diseases of the person gradually become fatal. Persons dealing with persons suspected of measles or scarlet fever, or those suffering from various infectious diseases, whom reviewers have heard called \" idiotic families,\" are included in this great category. But there are also those who",
+    "<|bos|>Charm's Tenements is large and rapid; The Cranes also move rapidly; The Diogenes moves quicker than the\n\nBox Tenements in Favor of Sponge.\"\n\nModernist odeists rapidly equip piants greatly in numbers,\"\n\nAmong the earliest in cison. At the present time they are in considerable numbers. Henle's Tenements (?) in 1891 appeared at a very high rate of advance. Storm increases the number of coaches annually.\n\nAt Blankham, Sauqs, Mouqs, and the French posts are large and ready to get in. Cesse is only about",
+    "<|bos|>Hires 350,752\n\nION.\n\n12 CH\n\nJUDGMENT\n\n9 I\n\nJUDGMENT\n\n11 I A\n\nCOLUMNS\n\nCALL OF HALF-HOW THAT HORN OF BEST PLEADNIKLI BONDS, HOLD SICKNESS, EPULTHESTA DAY, MACYDHIA 50 Enter DISPRISEPS, STONEOS, CONCORD HOME, SKETONING\n\nIt is the fair and pleasant Hibernia, Like sister well-pressed and pretty, that her pictures adorn;\n\nThus with her needle glean Sunday lies;\n\nOn ours that",
+    "<|bos|>eed\n\nCaswell paid them for the wheat, and seft i post gave a notice of their intention to pro- Scork had tenanted\n\nCrown Island, and lb-No, find it was for to imprison before taking affidavits, and offing great odds; and absent' private in Drj: Watts' time he had to come of post when in the spring before estate on Douglass lunged; have thought no time remained fifteen Months without complaining as to deposition on oath or affirmation;th Idum: could not find the housevantages the estate on Champlain that has the most of Ahembie the Sir=for",
+    "<|bos|>PREFACE\n\nand require. This. I can hardly accomplish. being able to fulfil my expressed purpose, and that of frankly acknowledging my indebtedness at least, but of asking for information of the furthest value, and at first glance. It is only in one place that most of this provision is inserted.\n\nA political architect might tire of the details of a Fish Commission much longer and often longer in Ireland, and could criticise expressions of reform from every-day affairs. We find, for example, in one of the National Oriental Libraries an interesting official examination of a Block building in all respects complete.\n\nIs there, in the Professor's evidence,",
+    "<|bos|>Clarke. He repeated the attempt to wrest Hitmarsh to be thrown into the proper place; whereupon there were subpoenas, enjoining such necessary preparations as a rushing wind could suggest. The latter was opposed; for a smell of burning sulphur in it did simply add to the trouble. Then, after wearing so many ropes about him, they were tied with ropes and drags of blue cloth and led away to Woburn, where it would have beene reserved for the mob to have turned their weapons into hempen strings of beads, and substituted clasp-knives and ropes like to pieces of wood for cloaks out of Plymouth",
+    "<|bos|>Hist. Ver.\n\n1 The rerew of a large domain, that gases absorbed and the tone of the plant in the use of the mineral matter overcoming large quantities\n\nof mineral matter are easily converted into the earth.\n\nIt is the man who constructs or makes a portion of the great mountain, and so fulfills his purpose. If, however, we look closer, and see how slowly and imperfectly the law of matter is acted upon by elective or experimentally easily effused forces in nature, then we can scarcely say the same things as when Nature is comprehended under her narrow but tremendous definition of beauty. It is an ideal revelation. In"
+  ]
+}

experiments/1930s-d12-r12-4096ctx-run2/evals/val_bpb.json

@@ -0,0 +1,12 @@
+{
+  "model": "base_model (step 2520)",
+  "step": 2520,
+  "bpb": {
+    "val": 1.0332056972909351
+  },
+  "core_metric": null,
+  "core_results": null,
+  "centered_results": null,
+  "conditioned_samples": [],
+  "unconditioned_samples": []
+}