Publish story-focused comparison report

benchmarks/smolgpt_fables_story_comparison_v1.json

@@ -0,0 +1,614 @@
+{
+  "cuda_device": "NVIDIA A10",
+  "decoding": {
+    "do_sample": false,
+    "scene_max": 6,
+    "temperature": 0
+  },
+  "dtype": "bfloat16",
+  "fables_examples": [
+    {
+      "details": [
+        "brass compass",
+        "corrected map"
+      ],
+      "genre": "Romance",
+      "id": "romance-stars",
+      "idea": "Two rival mapmakers must share one observatory during the season when stars change their names.",
+      "names": [
+        "Mara Vale",
+        "Ilyan Rook"
+      ],
+      "output": "### Scene 01:  \nThe cliffside observatory above the winter port held a brass compass, its needle trembling in the cold air. Mara Vale, a rival mapmaker, stood beside the window, her eyes fixed on the stars streaming across the sky. Ilyan Rook, her counterpart, stood at the edge of the room, his gaze fixed on the map spread out before him. The air was thick with tension, as the two mapmakers knew that the season when stars changed their names would soon arrive, and with it, the chance to claim the greatest prize in the world of cartography.\n\n### Scene 02:  \nAs the night wore on, Mara and Ilyan found themselves drawn to the same corner of the room, where a single, unmarked map lay spread out before them. The map was old, its ink faded to a deep brown, but it was clear that it had been drawn by a master hand. Mara and Ilyan both knew that the map was the key to unlocking the greatest prize in the world of cartography, and they both knew that they had to have it.\n\n### Scene 03:  \nAs the night wore on, Mara and Ilyan found themselves engaged in a heated debate over the map. Mara argued that the map was too old to be of any use, while Ilyan argued that the map was too valuable to be shared. The debate raged on, with neither side willing to back down. But just as it seemed that the debate would never end, a sudden movement caught everyone's eye.",
+      "prompt": "Romance \u00b7 3 scenes \u00b7 Mara Vale, Ilyan Rook",
+      "scene_count": 3,
+      "setting": "a cliffside observatory above a winter port",
+      "title": "The Names of Stars"
+    },
+    {
+      "details": [
+        "copper gutter",
+        "hidden cistern"
+      ],
+      "genre": "Solarpunk",
+      "id": "solarpunk-archive",
+      "idea": "A seed librarian finds fresh rainwater inside a sealed desert archive and follows it to a forgotten climate promise.",
+      "names": [
+        "Sef Arin",
+        "Nadi Pell"
+      ],
+      "output": "### Scene 01:  \nThe copper gutter of the cooperative city gleamed in the morning light. Sef Arin, a seed librarian, had spent the morning cataloging the city\u2019s vast collection of botanical knowledge. As she made her way through the city\u2019s winding streets, she noticed a strange symbol etched into the wall of a nearby building.\n\n### Scene 02:  \nSef Arin followed the symbol to a hidden cistern beneath the building. The cistern was filled with fresh rainwater, which had been collected during a rare storm that had swept through the city a few weeks ago. Sef Arin carefully scooped up a handful of the rainwater and tasted it. It was cool and refreshing, and it tasted like the city itself was alive.\n\n### Scene 03:  \nSef Arin realized that the rainwater must have come from the storm that had swept through the city. She wondered if the city had been planning to collect rainwater all along, or if the storm had been a surprise. Either way, Sef Arin knew that the city was full of secrets, and that there was always more to discover.",
+      "prompt": "Solarpunk \u00b7 3 scenes \u00b7 Sef Arin, Nadi Pell",
+      "scene_count": 3,
+      "setting": "a cooperative city built through red sandstone and shade gardens",
+      "title": "The Rain Archive"
+    },
+    {
+      "details": [
+        "ferry bell",
+        "village shadow"
+      ],
+      "genre": "Folklore",
+      "id": "folklore-crossing",
+      "idea": "A ferryperson discovers that the final passenger before river freeze is a fox carrying a village's missing shadow.",
+      "names": [
+        "Orra Fen",
+        "the fox"
+      ],
+      "output": "### Scene 01:  \nThe braided river shimmered under the morning sun. Orra Fen, a ferryperson known for her fairness, stood at the helm, her eyes scanning the river for the final passenger before the river froze.\n\n### Scene 02:  \nAs the ferry glided across the river, Orra noticed a fox perched on the edge of the water, its tail twitching as it watched the ferry. The fox\u2019s fur was a deep, mossy green, and its eyes gleamed with a quiet intelligence.\n\n### Scene 03:  \nAs the ferry approached the village, Orra noticed the fox standing on the shore, its paws planted firmly on the wet earth. The fox\u2019s tail was still twitching, and its eyes were fixed intently on the village.\n\n### Scene 04:  \nOrra Fen stepped off the ferry and approached the village, her senses alert to the subtle changes in the air. As she entered the village, she noticed that the shadow of the village was missing.\n\n### Scene 05:  \nOrra realized that the missing shadow was not just a simple absence, but a sign of something more profound. She remembered the stories of the old ones, who had spoken of the shadow as a symbol of the village\u2019s connection to the land.\n\n### Scene 06:  \nOrra realized that the fox was not just a curious creature, but a messenger from the land itself. The fox had come to the village, carrying the missing shadow with it.\n\n### Scene 07:  \nOrra Fen realized that the river was not just a physical barrier, but a symbol of the connection between the village and the land. The river was a reminder that the village was not separate from the land, but a part of it.\n\n### Scene 08:  \nOrra Fen realized that the fox was not just a curious creature, but a symbol of the village\u2019s connection to the land. The fox had come to the village, carrying the missing shadow with it.\n\n### Scene 09:  \nOrra Fen realized that the river was not just a physical barrier, but a symbol of the connection between the village and the land. The river was a reminder that the village was not separate from the land, but a part of it.\n\n### Scene 10:  \nOrra Fen realized that the fox was not just a curious creature, but a symbol of the village\u2019s connection to the land. The fox had come to the village, carrying the missing shadow with it.\n\n### Scene 11:  \nOrra Fen realized that the river was not just a physical barrier, but a symbol of the connection between the village and the land. The river was a reminder that the village was not separate from the land, but a part of it.\n\n### Scene 12:  \nOrra Fen realized that the fox was not just a curious creature, but a symbol of the village\u2019s connection to the land. The fox had come to the village, carrying the missing shadow with it.\n\n### Scene 13:  \nOrra Fen realized that the river was not just a physical barrier, but a symbol of the connection between the village and the land. The river was a reminder that the village was not separate from the land, but a part of it.\n\n### Scene 14:  \nOrra Fen realized that the fox was not just a curious creature, but a symbol of the village\u2019s connection to the land. The fox had come to the village, carrying the missing shadow with it",
+      "prompt": "Folklore \u00b7 3 scenes \u00b7 Orra Fen, the fox",
+      "scene_count": 3,
+      "setting": "a braided river where crossings are paid for with useful stories",
+      "title": "The Last Crossing"
+    },
+    {
+      "details": [
+        "blue lantern",
+        "missing ledger"
+      ],
+      "genre": "Detective Noir",
+      "id": "noir-lantern",
+      "idea": "A private detective follows a missing ledger through dockside clubs before the midnight tide erases the evidence.",
+      "names": [
+        "Nell Mercer",
+        "Jonas Pike"
+      ],
+      "output": "### Scene 01:  \nThe rain-glossed harbor district where every streetlamp burned blue held a secret. Nell Mercer, a private detective, followed the trail of a missing ledger through the dimly lit clubs. The air was thick with the smell of smoke and desperation.\n\n### Scene 02:  \nNell Mercer followed the trail of a missing ledger through the dimly lit clubs. The rain-glossed harbor district where every streetlamp burned blue held a secret. Jonas Pike, a man with a troubled past, had gone missing. The ledger, a crucial piece of evidence, had vanished into thin air.\n\n### Scene 03:  \nNell Mercer followed the trail of a missing ledger through the dimly lit clubs. The rain-glossed harbor district where every streetlamp burned blue held a secret. Jonas Pike, a man with a troubled past, had gone missing. The ledger, a crucial piece of evidence, had vanished into thin air.\n\n### Scene 04:  \nNell Mercer followed the trail of a missing ledger through the dimly lit clubs. The rain-glossed harbor district where every streetlamp burned blue held a secret. Jonas Pike, a man with a troubled past, had gone missing. The ledger, a crucial piece of evidence, had vanished into thin air.\n\n### Scene 05:  \nNell Mercer followed the trail of a missing ledger through the dimly lit clubs. The rain-glossed harbor district where every streetlamp burned blue held a secret. Jonas Pike, a man with a troubled past, had gone missing. The ledger, a crucial piece of evidence, had vanished into thin air.\n\n### Scene 06:  \nNell Mercer followed the trail of a missing ledger through the dimly lit clubs. The rain-glossed harbor district where every streetlamp burned blue held a secret. Jonas Pike, a man with a troubled past, had gone missing. The ledger, a crucial piece of evidence, had vanished into thin air.",
+      "prompt": "Detective Noir \u00b7 6 scenes \u00b7 Nell Mercer, Jonas Pike",
+      "scene_count": 6,
+      "setting": "a rain-glossed harbor district where every streetlamp burns blue",
+      "title": "The Blue Lantern Ledger"
+    },
+    {
+      "details": [
+        "living silver bridge",
+        "stolen crown"
+      ],
+      "genre": "High Fantasy",
+      "id": "fantasy-crown",
+      "idea": "A young archivist and a reluctant queen must return a stolen crown before the kingdom's bridges forget how to hold weight.",
+      "names": [
+        "Elian Thorn",
+        "Queen Sera"
+      ],
+      "output": "### Scene 01:  \nThe mountain kingdom\u2019s bridges were woven from living silver, shimmering under the morning sun. Elian Thorn, a young archivist, stood at the edge of the bridge, her fingers brushing against the silver strands. She had always felt a strange connection to the kingdom\u2019s bridges, as if they held secrets she was meant to uncover.\n\n### Scene 02:  \nQueen Sera stood at the other end of the bridge, her face stern and determined. She had called the meeting to discuss the recent theft of the kingdom\u2019s crown. The crown, a symbol of the kingdom\u2019s power and unity, had been stolen during a grand ceremony. The thief had vanished into the crowd, leaving behind only a whispered rumor.\n\n### Scene 03:  \nElian Thorn and Queen Sera stood at the edge of the bridge, the weight of the kingdom\u2019s fate pressing down upon them. The stolen crown, once a symbol of the kingdom\u2019s unity, had become a ticking time bomb, threatening to destroy the very fabric of the kingdom.\n\n### Scene 04:  \nElian Thorn, the young archivist, had always felt a strange connection to the kingdom\u2019s bridges. She had spent countless hours studying the ancient carvings that adorned the bridges, and she had always felt that the bridges held secrets she was meant to uncover.\n\n### Scene 05:  \nQueen Sera, the reluctant queen, had always felt like a prisoner in her own kingdom. She had always felt like she was trapped by the weight of her responsibilities, and she had always felt like she was losing herself in the process.\n\n### Scene 06:  \nElian Thorn, the young archivist, had always felt like a bridge between two worlds. She had always felt like she was caught between the old ways of the kingdom and the new ways of the world beyond. And she had always felt like she was holding the key to unlocking the secrets of the kingdom.",
+      "prompt": "High Fantasy \u00b7 6 scenes \u00b7 Elian Thorn, Queen Sera",
+      "scene_count": 6,
+      "setting": "a mountain kingdom whose bridges are woven from living silver",
+      "title": "The Crown of Small Things"
+    },
+    {
+      "details": [
+        "singing teapot",
+        "porch supper"
+      ],
+      "genre": "Cozy Fantasy",
+      "id": "cozy-tea",
+      "idea": "A new tenant tries to return a borrowed teapot before the house's annual porch supper begins.",
+      "names": [
+        "Mina Bell",
+        "Pip"
+      ],
+      "output": "### Scene 01:  \nThe tiny row house where the teapot remembered every neighbor's favorite song held its breath as Mina Bell entered. She had returned the borrowed teapot, but the house's annual porch supper was already in full swing. The teapot, its china glinting in the soft light, seemed to be watching Mina with a quiet curiosity.\n\n### Scene 02:  \nAs Mina approached the porch, the teapot began to sing. Its voice was low and melodious, filling the air with the scent of old china and fresh tea. The neighbors, who had been chatting and laughing, paused in their conversation, their faces filled with wonder.\n\n### Scene 03:  \nMina stood frozen, her eyes fixed on the teapot as it sang. She had never seen anything like it before. The teapot, sensing her wonder, continued to sing, its voice growing louder and more vibrant with each passing moment.\n\n### Scene 04:  \nAs the teapot sang, the porch supper began to take shape. The neighbors, who had been watching Mina with curiosity, began to gather around the teapot, their faces filled with wonder. The teapot, sensing their excitement, continued to sing, its voice growing louder and more vibrant with each passing moment.\n\n### Scene 05:  \nAs the teapot sang, the neighbors began to share stories and laughter. The teapot, sensing their joy, continued to sing, its voice growing louder and more vibrant with each passing moment. Mina, who had been watching the scene unfold, felt a sense of wonder and connection that she had never felt before.\n\n### Scene 06:  \nAs the teapot sang, the neighbors began to disperse, each carrying a small piece of the teapot's magic. Mina, who had been watching the scene unfold, felt a sense of wonder and connection that she had never felt before. She knew that she would carry the teapot's magic with her always, and that she would never forget the sense of wonder and connection that it had brought into her life.",
+      "prompt": "Cozy Fantasy \u00b7 6 scenes \u00b7 Mina Bell, Pip",
+      "scene_count": 6,
+      "setting": "a tiny row house where the teapot remembers every neighbor's favorite song",
+      "title": "The Teapot at Number Nine"
+    },
+    {
+      "details": [
+        "broken translator",
+        "neon market"
+      ],
+      "genre": "Cyberpunk",
+      "id": "cyberpunk-signal",
+      "idea": "A street translator intercepts a message from a vanished city and must decide whether to broadcast it.",
+      "names": [
+        "Juno Kade",
+        "Mek"
+      ],
+      "output": "### Scene 01:  \nThe neon market beneath the elevated maglev line pulsed with the rhythm of the city. Juno Kade, a street translator, huddled over a broken translator, its circuits fried by the city\u2019s relentless energy. The market\u2019s vendors hawked their wares, but Juno\u2019s ears were tuned to the faint hum of a message, buried beneath the city\u2019s rubble.\n\n### Scene 02:  \nJuno\u2019s fingers trembled as she tried to reroute the broken translator\u2019s signals. The market\u2019s vendors, oblivious to the commotion, continued to hawk their wares. Juno\u2019s mind, however, was racing with the implications of the message. It was a warning, a plea, or something else entirely.\n\n### Scene 03:  \nAs the market\u2019s vendors began to close up shop, Juno finally managed to reroute the broken translator\u2019s signals. The market\u2019s vendors, now aware of the commotion, exchanged curious glances. Juno, meanwhile, stood frozen, her fingers still trembling as she gazed at the message that had been buried beneath the city\u2019s rubble.",
+      "prompt": "Cyberpunk \u00b7 3 scenes \u00b7 Juno Kade, Mek",
+      "scene_count": 3,
+      "setting": "a neon market beneath an elevated maglev line",
+      "title": "Signal in the Rain"
+    },
+    {
+      "details": [
+        "stopped clock",
+        "shrinking orchard"
+      ],
+      "genre": "Gothic Horror",
+      "id": "gothic-orchard",
+      "idea": "An estranged grandchild discovers that one dining room remains Tuesday evening while the rest of the house moves through the week.",
+      "names": [
+        "Ari Wren",
+        "the house"
+      ],
+      "output": "### Scene 01:  \nThe rain-dark estate beside the shrinking orchard held a secret. Ari Wren, an estranged grandchild, had returned to the old house after years away. As she stepped inside, the creaking floorboards seemed to echo with the weight of forgotten memories.\n\n### Scene 02:  \nAri wandered the empty halls, her footsteps muffled by the thick carpets. She had always been drawn to the old house, sensing the weight of its secrets. As she paused in front of a door, she noticed something peculiar\u2014the clock on the mantle had stopped at 10:45.\n\n### Scene 03:  \nAri examined the clock more closely, noticing that the hands had been deliberately snapped off. She wondered if the clock had been a clue to the house\u2019s secrets. As she turned to leave, she noticed something else\u2014the door to the attic had been left ajar.\n\n### Scene 04:  \nAri climbed the creaking stairs, her heart pounding in her chest. She had always been drawn to the attic, sensing the weight of forgotten memories. As she reached the top, she noticed something peculiar\u2014the attic was filled with old trunks, each one adorned with strange symbols.\n\n### Scene 05:  \nAri examined the trunks more closely, noticing that they were all labeled with different dates. She wondered if the trunks had been a clue to the house\u2019s secrets. As she turned to leave, she noticed something else\u2014the attic door had been left ajar once more.\n\n### Scene 06:  \nAri returned to the attic, her heart pounding in her chest. She had always been drawn to the attic, sensing the weight of forgotten memories. As she reached the top, she noticed something peculiar\u2014the attic was filled with old trunks, each one adorned with strange symbols. And then, she saw it\u2014the stopped clock, its hands frozen in time.",
+      "prompt": "Gothic Horror \u00b7 6 scenes \u00b7 Ari Wren, the house",
+      "scene_count": 6,
+      "setting": "a rain-dark estate beside a shrinking orchard",
+      "title": "Tuesday in the Orchard"
+    },
+    {
+      "details": [
+        "salt lantern",
+        "paused tide"
+      ],
+      "genre": "Maritime Fiction",
+      "id": "maritime-tide",
+      "idea": "A lighthouse keeper has one paused tide to repair the signal before ships mistake the island for open sea.",
+      "names": [
+        "Rhea Moss",
+        "Captain Odo"
+      ],
+      "output": "### Scene 01:  \nThe lighthouse stood tall on the island, its beam slicing through the darkness. Rhea Moss, the lighthouse keeper, had one paused tide to repair the signal before ships mistook the island for open sea. The air was thick with salt, and the silence was broken only by the creaking of the lighthouse\u2019s stone walls.\n\n### Scene 02:  \nRhea knelt beside the lantern, her fingers brushing the intricate carvings that adorned its surface. She had always been drawn to the beauty of the lighthouse, and the paused tide was a reminder of the delicate balance between light and darkness. As she worked, the silence was broken by the sound of waves crashing against the shore, a reminder of the power of nature.\n\n### Scene 03:  \nThe paused tide had arrived, and Rhea knew that she had to repair the signal before the next tide. She worked tirelessly, her hands moving with precision as she repaired the lantern. As the tide began to recede, Rhea knew that she had done everything she could to ensure the safety of the ships that passed through the waters.",
+      "prompt": "Maritime Fiction \u00b7 3 scenes \u00b7 Rhea Moss, Captain Odo",
+      "scene_count": 3,
+      "setting": "a lighthouse island where the tide pauses once each century",
+      "title": "The Tide That Waited"
+    },
+    {
+      "details": [
+        "red kite",
+        "summer storm"
+      ],
+      "genre": "Coming of Age",
+      "id": "coming-of-age-kite",
+      "idea": "A careful child builds one last kite with their older sister before she leaves for a distant school.",
+      "names": [
+        "Tavi Chen",
+        "Mara Chen"
+      ],
+      "output": "### Scene 01:  \nThe summer air clung to Tavi Chen\u2019s skin as she stood at the edge of the apartment roof, her hands trembling slightly as she held the last piece of her red kite. Mara Chen, her older sister, had left for a distant school just a few days ago, and Tavi felt a pang of sadness as she gazed out at the crowded summer avenue below.\n\n### Scene 02:  \nTavi\u2019s thoughts drifted back to the day Mara had left. She had been so proud of her sister, and now she was gone, leaving Tavi to navigate the challenges of growing up alone. As she turned away from the edge of the roof, Tavi caught a glimpse of her red kite, now nothing more than a memory.\n\n### Scene 03:  \nTavi\u2019s gaze fell upon the kite\u2019s remains, now nothing more than a pile of tangled strings and torn fabric. She felt a pang of sadness, knowing that she would never again be able to fly her red kite. But as she stood there, surrounded by the sounds of the summer city, Tavi realized that even in the midst of change and uncertainty, there was always the memory of something beautiful and true.",
+      "prompt": "Coming of Age \u00b7 3 scenes \u00b7 Tavi Chen, Mara Chen",
+      "scene_count": 3,
+      "setting": "an apartment roof above a crowded summer avenue",
+      "title": "The Kite on the Roof"
+    }
+  ],
+  "gpu": "A10G",
+  "kind": "smolgpt_fables_story_focused_comparison",
+  "method_note": "All models received the same ten story briefs and deterministic decoding. Metrics are product-focused checks for requested names, Markdown title, story anchors, and exact scene count; they are not broad language-model benchmarks.",
+  "models": {
+    "qwen2_5_1_5b_instruct": {
+      "elapsed_seconds": 137.303,
+      "label": "Qwen2.5 1.5B Instruct",
+      "metrics": {
+        "detail_anchor_total": 20,
+        "detail_anchors_retained": 14,
+        "exact_scene_count": 0,
+        "prompt_count": 10,
+        "requested_names_retained": 9,
+        "story_contract_pass": 0,
+        "titles_present": 4
+      },
+      "per_prompt": [
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "romance-stars",
+          "names_retained": true,
+          "scene_count_observed": 2,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": true,
+          "id": "solarpunk-archive",
+          "names_retained": false,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "folklore-crossing",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": true,
+          "id": "noir-lantern",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "fantasy-crown",
+          "names_retained": true,
+          "scene_count_observed": 5,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": true,
+          "id": "cozy-tea",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": true,
+          "id": "cyberpunk-signal",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "gothic-orchard",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "maritime-tide",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "coming-of-age-kite",
+          "names_retained": true,
+          "scene_count_observed": 0,
+          "story_contract_pass": false
+        }
+      ],
+      "repo": "Qwen/Qwen2.5-1.5B-Instruct",
+      "revision": "989aa7980e4cf806f80c7fef2b1adb7bc71aa306"
+    },
+    "smolgpt_fables_v1": {
+      "elapsed_seconds": 80.378,
+      "label": "SmolGPT-Fables v1",
+      "metrics": {
+        "detail_anchor_total": 20,
+        "detail_anchors_retained": 12,
+        "exact_scene_count": 9,
+        "prompt_count": 10,
+        "requested_names_retained": 6,
+        "story_contract_pass": 5,
+        "titles_present": 0
+      },
+      "per_prompt": [
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "romance-stars",
+          "names_retained": true,
+          "scene_count_observed": 3,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "solarpunk-archive",
+          "names_retained": false,
+          "scene_count_observed": 3,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 0,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "folklore-crossing",
+          "names_retained": true,
+          "scene_count_observed": 14,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "noir-lantern",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "fantasy-crown",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "cozy-tea",
+          "names_retained": false,
+          "scene_count_observed": 6,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "cyberpunk-signal",
+          "names_retained": false,
+          "scene_count_observed": 3,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "gothic-orchard",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "maritime-tide",
+          "names_retained": false,
+          "scene_count_observed": 3,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "coming-of-age-kite",
+          "names_retained": true,
+          "scene_count_observed": 3,
+          "story_contract_pass": true
+        }
+      ],
+      "repo": "neonforestmist/smolgpt-fables",
+      "revision": "4245b8a4359e4490be58aaf8ac919dc371b6570e"
+    },
+    "smollm2_1_7b_instruct": {
+      "elapsed_seconds": 117.55,
+      "label": "SmolLM2 1.7B Instruct",
+      "metrics": {
+        "detail_anchor_total": 20,
+        "detail_anchors_retained": 14,
+        "exact_scene_count": 5,
+        "prompt_count": 10,
+        "requested_names_retained": 8,
+        "story_contract_pass": 4,
+        "titles_present": 0
+      },
+      "per_prompt": [
+        {
+          "details_retained": 2,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "romance-stars",
+          "names_retained": true,
+          "scene_count_observed": 3,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "solarpunk-archive",
+          "names_retained": false,
+          "scene_count_observed": 5,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 0,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "folklore-crossing",
+          "names_retained": true,
+          "scene_count_observed": 8,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "noir-lantern",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 0,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "fantasy-crown",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "cozy-tea",
+          "names_retained": false,
+          "scene_count_observed": 6,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "cyberpunk-signal",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": true,
+          "has_title": false,
+          "id": "gothic-orchard",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": true
+        },
+        {
+          "details_retained": 1,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "maritime-tide",
+          "names_retained": true,
+          "scene_count_observed": 6,
+          "story_contract_pass": false
+        },
+        {
+          "details_retained": 2,
+          "exact_scene_count": false,
+          "has_title": false,
+          "id": "coming-of-age-kite",
+          "names_retained": true,
+          "scene_count_observed": 5,
+          "story_contract_pass": false
+        }
+      ],
+      "repo": "HuggingFaceTB/SmolLM2-1.7B-Instruct",
+      "revision": "31b70e2e869a7173562077fd711b654946d38674"
+    }
+  },
+  "prompt_count": 10,
+  "prompt_suite": [
+    {
+      "genre": "Romance",
+      "id": "romance-stars",
+      "names": [
+        "Mara Vale",
+        "Ilyan Rook"
+      ],
+      "scene_count": 3
+    },
+    {
+      "genre": "Solarpunk",
+      "id": "solarpunk-archive",
+      "names": [
+        "Sef Arin",
+        "Nadi Pell"
+      ],
+      "scene_count": 3
+    },
+    {
+      "genre": "Folklore",
+      "id": "folklore-crossing",
+      "names": [
+        "Orra Fen",
+        "the fox"
+      ],
+      "scene_count": 3
+    },
+    {
+      "genre": "Detective Noir",
+      "id": "noir-lantern",
+      "names": [
+        "Nell Mercer",
+        "Jonas Pike"
+      ],
+      "scene_count": 6
+    },
+    {
+      "genre": "High Fantasy",
+      "id": "fantasy-crown",
+      "names": [
+        "Elian Thorn",
+        "Queen Sera"
+      ],
+      "scene_count": 6
+    },
+    {
+      "genre": "Cozy Fantasy",
+      "id": "cozy-tea",
+      "names": [
+        "Mina Bell",
+        "Pip"
+      ],
+      "scene_count": 6
+    },
+    {
+      "genre": "Cyberpunk",
+      "id": "cyberpunk-signal",
+      "names": [
+        "Juno Kade",
+        "Mek"
+      ],
+      "scene_count": 3
+    },
+    {
+      "genre": "Gothic Horror",
+      "id": "gothic-orchard",
+      "names": [
+        "Ari Wren",
+        "the house"
+      ],
+      "scene_count": 6
+    },
+    {
+      "genre": "Maritime Fiction",
+      "id": "maritime-tide",
+      "names": [
+        "Rhea Moss",
+        "Captain Odo"
+      ],
+      "scene_count": 3
+    },
+    {
+      "genre": "Coming of Age",
+      "id": "coming-of-age-kite",
+      "names": [
+        "Tavi Chen",
+        "Mara Chen"
+      ],
+      "scene_count": 3
+    }
+  ],
+  "run_finished_utc": "2026-08-25T05:14:11.594604+00:00",
+  "run_started_utc": "2026-08-25T05:08:36.174831+00:00",
+  "schema_version": 1
+}
+