Upload evals for think-d12-1ep-65sh-r30

experiments/think-d12-1ep-65sh-r30/evals/core.json

@@ -0,0 +1,56 @@
+{
+  "model": "base_model (step 6300)",
+  "step": 6300,
+  "bpb": {},
+  "core_metric": 0.08955908817565156,
+  "core_results": {
+    "hellaswag_zeroshot": 0.2850029766559601,
+    "jeopardy": 0.0,
+    "bigbench_qa_wikidata": 0.09463116526603699,
+    "arc_easy": 0.32575756311416626,
+    "arc_challenge": 0.20392490923404694,
+    "copa": 0.5899999737739563,
+    "commonsense_qa": 0.31859132647514343,
+    "piqa": 0.5631120800971985,
+    "openbook_qa": 0.23400001227855682,
+    "lambada_openai": 0.23345623910427094,
+    "hellaswag": 0.2857000529766083,
+    "winograd": 0.5494505763053894,
+    "winogrande": 0.5114443302154541,
+    "bigbench_dyck_languages": 0.11000000685453415,
+    "agi_eval_lsat_ar": 0.27391302585601807,
+    "bigbench_cs_algorithms": 0.40984848141670227,
+    "bigbench_operators": 0.095238097012043,
+    "bigbench_repeat_copy_logic": 0.0,
+    "squad": 0.047209080308675766,
+    "coqa": 0.09420017153024673,
+    "boolq": 0.5911315083503723,
+    "bigbench_language_identification": 0.25509998202323914
+  },
+  "centered_results": {
+    "hellaswag_zeroshot": 0.04667063554128011,
+    "jeopardy": 0.0,
+    "bigbench_qa_wikidata": 0.09463116526603699,
+    "arc_easy": 0.10101008415222168,
+    "arc_challenge": -0.06143345435460409,
+    "copa": 0.1799999475479126,
+    "commonsense_qa": 0.14823915809392926,
+    "piqa": 0.12622416019439697,
+    "openbook_qa": -0.021333316961924236,
+    "lambada_openai": 0.23345623910427094,
+    "hellaswag": 0.0476000706354777,
+    "winograd": 0.09890115261077881,
+    "winogrande": 0.022888660430908203,
+    "bigbench_dyck_languages": 0.11000000685453415,
+    "agi_eval_lsat_ar": 0.09239128232002257,
+    "bigbench_cs_algorithms": 0.40984848141670227,
+    "bigbench_operators": 0.095238097012043,
+    "bigbench_repeat_copy_logic": 0.0,
+    "squad": 0.047209080308675766,
+    "coqa": 0.09420017153024673,
+    "boolq": -0.07596971486744127,
+    "bigbench_language_identification": 0.18052803302886594
+  },
+  "conditioned_samples": [],
+  "unconditioned_samples": []
+}

experiments/think-d12-1ep-65sh-r30/evals/samples.json

@@ -0,0 +1,48 @@
+{
+  "model": "base_model (step 6300)",
+  "step": 6300,
+  "bpb": {},
+  "core_metric": null,
+  "core_results": null,
+  "centered_results": null,
+  "conditioned_samples": [
+    {
+      "prompt": "The capital of France is",
+      "text": "<|bos|>The capital of France is the capital of the world. \n\nThe capital of the world is the capital of"
+    },
+    {
+      "prompt": "The chemical symbol of gold is",
+      "text": "<|bos|>The chemical symbol of gold is the same as that of silver, and the same as that of copper. The"
+    },
+    {
+      "prompt": "If yesterday was Friday, then tomorrow will be",
+      "text": "<|bos|>If yesterday was Friday, then tomorrow will be Saturday. \n\nI am, dear Sir, Your most obedient servant,\n\nJ."
+    },
+    {
+      "prompt": "The opposite of hot is",
+      "text": "<|bos|>The opposite of hot is hot, and the hot is cold. \n\nThe hot is hot, and the"
+    },
+    {
+      "prompt": "The planets of the solar system are:",
+      "text": "<|bos|>The planets of the solar system are: \n\n1. The sun, the moon, and the stars. \n\n2."
+    },
+    {
+      "prompt": "My favorite color is",
+      "text": "<|bos|>My favorite color is the white, and the white is the color of the black. \n\nThe white"
+    },
+    {
+      "prompt": "If 5*x + 3 = 13, then x is",
+      "text": "<|bos|>If 5*x + 3 = 13, then x is the number of the first pair of eyes, and x is the number of the"
+    }
+  ],
+  "unconditioned_samples": [
+    "<|bos|>LEES, BARNS & W sale,12R.\n\nLONDON: I. SONS, 68-72 MARKET STREET; LEEDS,-PL ZAN Bauth & Co., 30 WALNUT STREET; ME., 1-69; \n\n12-70 Market STREET.\n\nEduc 7622.11 Vennifths \n\nCOLLEGE HARVARD HAR HAR LIBRARY FROM RR.Harov to 1913 \n\nUnivied by fiv end.\n\nman, Latham; The Cabbies' Tales, Raiders, Experiences, Myters and Other \n\nMarks, Andes); Hymns, Odd Fellows, Odd \n\nFellows",
+    "<|bos|>ily engaged in common domestic life again, until, weary of living in tents, restricted within narrow bounds, at the ATLstock farm, where they were accustomed to meet and converse in common, it was too much for them to expect to endure ichkosisa realities; they became aware of the fact, and having come together for worship, it was a melodramatic performance.\n\nThe Inviciosorum closer system. \n\nAnd now, humous clay workings, whose steep walls have stood the free\u03c4\u03af\u03b6\u03c9\u03bd into the world for so many hundreds of generations, begin to rise with the dirt instigated by the witch",
+    "<|bos|>HARVARD Edinburgh, and approved of Bland in presenting certain designs which he designed for queen's theatres and building churches, or accomplished very singular effects in external art as man served us, and concerning which we the English rose musically, and saidtherefore like most of her modernit say, probably from the bust of Miss V., and endeavour to praise the happy conduct of the French' pacificists, in being able to perpetuat, and within her theatre, towards this ment, as this very compliment was used to her with talent, though in lists for every day relative liberally; for, her night-attentresses. \n\n",
+    "<|bos|>Oliver ORmonds. \n\n666.] A river may cut a dozen bars of iron of different forms and shapes; it may do so with sundry branches, and not with one uniform hard iron line or one uniform thickness; it may construct a timber stump of a jutting sort of oaken beam, or of eights, cylinders, or 25\"4 square inches extending from two pine branches, upon one road-stone, in one frame and without frame, and then 3\" through, etc.\"-Walter Bevis. \n\n592.] No intermediate plank, or wrought-iron frame at all :- -",
+    "<|bos|> \u05d9\u05d4 \u05d5\u05d4 \n\nINSTITVTIO THEOLOGICA ANDOVER FVNDATA MDCCCVII \n\n\u0391\u039a\u03a1\u039f\u0393\u03a9\u039d \n\nPs.CXIX JOH.XVII. \n\n179. oyocate\u2758 in- \u05d4\u05e0\u05d5\u05d3\u05d9 \u03cc\u03c3\u03bf\u03c2 \n\nChairs in Ps.CXIX JOH.XVII. \n\n169. \u05db\u05e8\u05db\u05e8\u05da $ 5.00\n\nMy address to Esther.\n\nY. H. C.\n\nJAMES STEPHEN Jeannette \n\nRichard Bladen, Onkelos' own act-died in 1863.\n\nJAMES STEPHEN Jeannette, York which he left to be the issue of the marriage of General Graham with Julia Morgan (my brother-in-law, the countess of \n\nB",
+    "<|bos|>alogue be transacted. The first instant vessel approximating to loading at the moment of port signals should be shaped.\" Her heel must be anchored relatively low in the snow.\" \n\nSo soon as \"papers to sell\" may claim permission to enter ports, Livingston refused to organise the commercialering business.\" \"Syrup of violets and pinks,\u201d adds Fairhaven, \"are infrequent people, and are not much used by those that live in cities.\"7 \"Darting in Position,\"  \"ida every thing except its periodical patterns.\" \"The pace and The smooth back guile",
+    "<|bos|> cemetery mills. . I sunk the Hudson's\n\nRiver steamer at Hamilton, New York, and telegraphed to the chairman of the board of directors 210 articles; disposed of by me at 100 each; disposed of at 50; only one lot taken off by accident occurred, and some irritated by language supposed to have been used! A skirmish occurred near Schenectady, N. Y., between the parties at 50; Grant, in person, made one of the battles; \n\nHal The Better Hill since called Home Hill; was the battle ground of General to-day.\n\nI was in the",
+    "<|bos|>ventions extensive Government insularffitho-anatolaruced.\n\nWhen we were still intestinely in the midst of #1 \n\nMahyrr' and not surprising the Oles they 460 this family | and the most eran wide Simpson.HISTORY AND \n\nour sety 18sn had magst hon and aour way if fagles Su8 f the | led their formed 66 must maintain she cler that the hand of iod st bomb e ideparting the une art ells buso ear and guts instde tep Our puff eth like the Jachers"
+  ]
+}

experiments/think-d12-1ep-65sh-r30/evals/val_bpb.json

@@ -0,0 +1,12 @@
+{
+  "model": "base_model (step 6300)",
+  "step": 6300,
+  "bpb": {
+    "val": 0.9944563924677391
+  },
+  "core_metric": null,
+  "core_results": null,
+  "centered_results": null,
+  "conditioned_samples": [],
+  "unconditioned_samples": []
+}