Expand native comparison to ten complex prompts

README.md

@@ -77,13 +77,46 @@ The compact benchmark summary records native BF16-vs-OrbitQuant evidence for the
 | Evidence | Value |
 | --- | --- |
 | Comparison matrix | `assets/image_generation_comparison_matrix.webp` |
-| Paired prompt/seed count | `1` |
-| BF16 source generated samples | `1` |
+| Paired prompt/seed count | `10` |
+| BF16 source generated samples | `10` |
 | BF16 source generated frames | `0` |
-| BF16 source nonempty outputs | `1` |
-| OrbitQuant generated samples | `1` |
+| BF16 source nonempty outputs | `10` |
+| OrbitQuant generated samples | `10` |
 | OrbitQuant generated frames | `0` |
-| OrbitQuant nonempty outputs | `1` |
+| OrbitQuant nonempty outputs | `10` |
+
+## Comparison Prompt Set
+
+The matrix uses all ten prompts from `image_visual_v2` at `1024x1024`. BF16 and OrbitQuant use the same seed for each row.
+
+| # | Stress case | Exact required text |
+| ---: | --- | --- |
+| 1 | 01 Fine-detail astrolabe | - |
+| 2 | 02 Layered character composition | - |
+| 3 | 03 Exact counting and choreography | - |
+| 4 | 04 Dense color and object binding | - |
+| 5 | 05 Nested spatial relationships | - |
+| 6 | 06 Cinematic night-market panorama | - |
+| 7 | 07 Editorial Latin typography | ORBIT QUANT; DATA WITHOUT CALIBRATION |
+| 8 | 08 Russian Constructivist typography | КВАНТОВАЯ ОРБИТА; МОСКВА 2049; КВАНТОВАНИЕ |
+| 9 | 09 Japanese typography and mixed style | 量子の軌道; 東京の未来 |
+| 10 | 10 Chinese typography, reflection, occlusion | 量子轨道; 未来之城 |
+
+<details>
+<summary>Exact comparison prompts</summary>
+
+1. **01 Fine-detail astrolabe**: A museum-grade macro photograph of a single ornate brass astronomical clock, covered with interlocking gears, engraved constellations, enamel moon phases, hair-thin hands, tiny screws, worn gilt edges, and dust caught in the mechanisms; dramatic Rembrandt lighting, black velvet background, extreme material detail, large-format photography, every cog mechanically coherent
+2. **02 Layered character composition**: A lacquered white android and an elderly watchmaker facing each other across a crowded Art Nouveau workshop, jointly repairing a mechanical hummingbird; the android is on the left, the watchmaker on the right, the bird centered between them, hundreds of tools and clock parts in the midground, rain and a tram visible through the window, coherent mirror reflections, cinematic tungsten and cyan light, intricate faces and hands, editorial realism
+3. **03 Exact counting and choreography**: Exactly seven masked dancers performing on seven separate illuminated platforms inside a vast flooded opera house, no extra dancers; alternating crimson and ivory costumes from left to right, each dancer in a distinct pose, balconies reflected in the water, floating candles, volumetric stage haze, baroque theatrical photography, sharp foreground and readable background architecture
+4. **04 Dense color and object binding**: An elaborate surreal fashion tableau with three models: the left model wears a cobalt-blue coat and holds a yellow glass sphere, the center model wears a saffron dress and holds a green ceramic pyramid, the right model wears an emerald suit and holds a red velvet cube; rococo greenhouse, rare orchids, patterned tile floor, prismatic sunlight, magazine-cover precision, preserve every color-object pairing
+5. **05 Nested spatial relationships**: A meticulous cutaway diorama of a vertical city: a glass greenhouse sits directly above a silver subway car, a red fox stands inside the greenhouse beneath a hanging moon lamp, a violinist waits below the subway platform, and a yellow airship passes behind the entire structure; isometric perspective, architectural-section drawing mixed with photoreal materials, dozens of tiny rooms, stairs and people, clean depth
+6. **06 Cinematic night-market panorama**: A sweeping cinematic panorama of a rain-soaked floating night market at blue hour: in the foreground a chef plates translucent dumplings under a red silk canopy; in the midground children chase paper lanterns across narrow bridges while merchants unload exotic fruit from wooden boats; in the background a terraced megacity rises through mist beneath a storm, with hundreds of warm windows, wet reflections, steam, umbrellas, ropes and signage; deep focus, anamorphic highlights, realistic faces, coherent perspective, painterly color grading with documentary-level detail
+7. **07 Editorial Latin typography**: A sophisticated Swiss International Style exhibition poster photographed behind slightly reflective museum glass, with the exact large headline "ORBIT QUANT" and the exact smaller subtitle "DATA WITHOUT CALIBRATION"; strict modular grid, red, black and white screenprint, tiny registration marks, embossed paper fibers, sharp letterforms, dramatic gallery shadows, no other text
+8. **08 Russian Constructivist typography**: A richly detailed Russian Constructivist science-fiction poster with the exact Cyrillic headline "КВАНТОВАЯ ОРБИТА", the exact subtitle "МОСКВА 2049", and a small exact stamp "КВАНТОВАНИЕ"; diagonal red and black geometry, cream paper, cosmonaut portrait, orbital diagrams, halftone grain, folded corners, layered ink, museum archival photograph, all letters crisp and correctly ordered
+9. **09 Japanese typography and mixed style**: An elaborate Japanese art magazine cover combining Edo woodblock printing with a futuristic Tokyo skyline, with the exact vertical title "量子の軌道" and the exact subtitle "東京の未来"; giant indigo waves curl around glass towers, red-crowned cranes cross a gold moon, tiny pedestrians and trains fill the lower streets, visible washi fibers, layered spot colors, precise Japanese glyphs, balanced editorial layout
+10. **10 Chinese typography, reflection, occlusion**: A luxurious Chinese retro-futurist department-store window at night with the exact gold title "量子轨道" and the exact red subtitle "未来之城"; a curved chrome robot is partly occluded by peonies and blue-and-white porcelain, the calligraphy and neon street must appear coherently reflected across its body, multiple glass layers, silk textures, passing bicycles, cinematic rain, fine product-photography detail
+
+</details>
 
 ## Quantization
 

SHA256SUMS

@@ -1,10 +1,10 @@
-4ffa9bfde4784613076fb15188361e5fa46139feb34aef72aa73512cb7309ebe  README.md
-61daee1980a75e3290a75598e2ec5db40b8841772bded7e2d54c7354ffa12fa9  assets/image_generation_comparison_matrix.webp
-c855f4ffaca8a20243c618e65870f948ca64707a8f8c4ef3bc6b4d789753a94c  benchmark/summary.json
+af3d32cb1d024d075fa7691e2b57b98ff87c565c7d5784f7e9c5e7aaf2682b5d  README.md
+0b036acba67d919d8bea0a073838344ea9c600d8d77702c904b86efc544f726c  assets/image_generation_comparison_matrix.webp
+b8b7c5a6a34ccb0fe3099ed0ba578505924ade93ba1fee46dc9b75b36169c6e0  benchmark/summary.json
 677ef7a12e36dd92731ba9973b1b31eff3e3f03ff432a33ed6cbd127dddf16eb  model.safetensors
 aa71fb7274ee17e5e94ff2434ec0aaa586bc74d09f077b5740ce4fb1a0af17c9  model_index.json
 80b16b36294630315e4c94a85e35b90d10b67e502cc1cd70bc7151fa44ff7a12  orbitquant_codebooks.safetensors
-65a6dcc8bd2a038b8c236922a28155ae4782d64255756ef913873483728c964f  orbitquant_manifest.json
+3385810d70fbb114ebe754a251309bd63e5d4f27f0c840d9b5c63cd1b7d6e9c9  orbitquant_manifest.json
 e1724c9394bf83ea05da6b457b8072d2bcb3d5a533dafce916da8deb517d3825  orbitquant_rotations.safetensors
-65ec8afff39a6cf80feee2eac2f06fa66e51cd471a8382f4355a04fd74eda0af  prompts.json
+0aee42e64abd66ff9293ec140df94e62bf7836eb96a21528a728282a0278c6ed  prompts.json
 afa4f2ede51d2f322a89ef593fbf50a6bdd9192bcfe160f6c5c77b4fe0117a3a  quantization_config.json

benchmark/summary.json

@@ -1,5 +1,5 @@
 {
-  "status": "not_run",
+  "status": "native_validation_complete",
   "source_model_id": "Tongyi-MAI/Z-Image-Turbo",
   "source_revision": "f332072aa78be7aecdf3ee76d5c247082da564a6",
   "weight_bits": 4,
@@ -21,33 +21,132 @@
   "metrics": {},
   "native_smoke": {
     "proof_format": "orbitquant-native-smoke-v1",
-    "proof_source": "local_compare_native_bundle",
+    "proof_source": "native_visual_pack_v2",
     "comparison_asset_path": "assets/image_generation_comparison_matrix.webp",
-    "paired_prompt_seed_count": 1,
+    "paired_prompt_seed_count": 10,
     "paired_prompt_seed_keys": [
       [
         "z-image-native",
         "0",
         "simple-object"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "two-object-composition"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "counting"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "color-binding"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "spatial-relationship"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "long-prompt"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "english-text-rendering"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "cyrillic-text-rendering"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "style-heavy"
+      ],
+      [
+        "z-image-native",
+        "0",
+        "occlusion-reflection"
       ]
     ],
     "splits": {
       "original": {
-        "records": 1,
-        "generated_samples": 1,
+        "records": 10,
+        "generated_samples": 10,
         "generated_frames": 0,
-        "nonempty_output_count": 1,
+        "nonempty_output_count": 10,
         "seeds": [
           "0"
         ],
         "prompt_ids": [
-          "simple-object"
+          "simple-object",
+          "two-object-composition",
+          "counting",
+          "color-binding",
+          "spatial-relationship",
+          "long-prompt",
+          "english-text-rendering",
+          "cyrillic-text-rendering",
+          "style-heavy",
+          "occlusion-reflection"
         ],
         "pair_keys": [
           [
             "z-image-native",
             "0",
             "simple-object"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "two-object-composition"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "counting"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "color-binding"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "spatial-relationship"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "long-prompt"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "english-text-rendering"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "cyrillic-text-rendering"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "style-heavy"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "occlusion-reflection"
           ]
         ],
         "native_settings": [
@@ -62,21 +161,75 @@
         ]
       },
       "orbitquant": {
-        "records": 1,
-        "generated_samples": 1,
+        "records": 10,
+        "generated_samples": 10,
         "generated_frames": 0,
-        "nonempty_output_count": 1,
+        "nonempty_output_count": 10,
         "seeds": [
           "0"
         ],
         "prompt_ids": [
-          "simple-object"
+          "simple-object",
+          "two-object-composition",
+          "counting",
+          "color-binding",
+          "spatial-relationship",
+          "long-prompt",
+          "english-text-rendering",
+          "cyrillic-text-rendering",
+          "style-heavy",
+          "occlusion-reflection"
         ],
         "pair_keys": [
           [
             "z-image-native",
             "0",
             "simple-object"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "two-object-composition"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "counting"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "color-binding"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "spatial-relationship"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "long-prompt"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "english-text-rendering"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "cyrillic-text-rendering"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "style-heavy"
+          ],
+          [
+            "z-image-native",
+            "0",
+            "occlusion-reflection"
           ]
         ],
         "native_settings": [
@@ -93,5 +246,6 @@
     }
   },
   "published_summary": "compact",
-  "raw_generation_records": "local-only"
+  "raw_generation_records": "local-only",
+  "comparison_prompt_pack": "image_visual_v2"
 }

orbitquant_manifest.json

@@ -2721,13 +2721,13 @@
     ]
   },
   "checksums": {
-    "assets/image_generation_comparison_matrix.webp": "61daee1980a75e3290a75598e2ec5db40b8841772bded7e2d54c7354ffa12fa9",
-    "benchmark/summary.json": "c855f4ffaca8a20243c618e65870f948ca64707a8f8c4ef3bc6b4d789753a94c",
+    "assets/image_generation_comparison_matrix.webp": "0b036acba67d919d8bea0a073838344ea9c600d8d77702c904b86efc544f726c",
+    "benchmark/summary.json": "b8b7c5a6a34ccb0fe3099ed0ba578505924ade93ba1fee46dc9b75b36169c6e0",
     "model.safetensors": "677ef7a12e36dd92731ba9973b1b31eff3e3f03ff432a33ed6cbd127dddf16eb",
     "model_index.json": "aa71fb7274ee17e5e94ff2434ec0aaa586bc74d09f077b5740ce4fb1a0af17c9",
     "orbitquant_codebooks.safetensors": "80b16b36294630315e4c94a85e35b90d10b67e502cc1cd70bc7151fa44ff7a12",
     "orbitquant_rotations.safetensors": "e1724c9394bf83ea05da6b457b8072d2bcb3d5a533dafce916da8deb517d3825",
-    "prompts.json": "65ec8afff39a6cf80feee2eac2f06fa66e51cd471a8382f4355a04fd74eda0af",
+    "prompts.json": "0aee42e64abd66ff9293ec140df94e62bf7836eb96a21528a728282a0278c6ed",
     "quantization_config.json": "afa4f2ede51d2f322a89ef593fbf50a6bdd9192bcfe160f6c5c77b4fe0117a3a"
   }
 }

prompts.json

@@ -1,57 +1,67 @@
 {
-  "prompt_pack": "image_visual_v1",
+  "prompt_pack": "image_visual_v2",
   "media_type": "image",
   "target_policy": "z_image",
   "prompts": [
     {
       "id": "simple-object",
-      "category": "simple_object",
-      "prompt": "A red ceramic mug on a wooden desk, soft daylight, shallow depth of field"
+      "title": "01 Fine-detail astrolabe",
+      "category": "fine_detail",
+      "prompt": "A museum-grade macro photograph of a single ornate brass astronomical clock, covered with interlocking gears, engraved constellations, enamel moon phases, hair-thin hands, tiny screws, worn gilt edges, and dust caught in the mechanisms; dramatic Rembrandt lighting, black velvet background, extreme material detail, large-format photography, every cog mechanically coherent"
     },
     {
       "id": "two-object-composition",
-      "category": "two_object_composition",
-      "prompt": "Two robots playing chess in a quiet library, one silver and one matte black"
+      "title": "02 Layered character composition",
+      "category": "layered_character_composition",
+      "prompt": "A lacquered white android and an elderly watchmaker facing each other across a crowded Art Nouveau workshop, jointly repairing a mechanical hummingbird; the android is on the left, the watchmaker on the right, the bird centered between them, hundreds of tools and clock parts in the midground, rain and a tram visible through the window, coherent mirror reflections, cinematic tungsten and cyan light, intricate faces and hands, editorial realism"
     },
     {
       "id": "counting",
-      "category": "counting",
-      "prompt": "Five glass marbles arranged in a straight line on white paper"
+      "title": "03 Exact counting and choreography",
+      "category": "counting_and_choreography",
+      "prompt": "Exactly seven masked dancers performing on seven separate illuminated platforms inside a vast flooded opera house, no extra dancers; alternating crimson and ivory costumes from left to right, each dancer in a distinct pose, balconies reflected in the water, floating candles, volumetric stage haze, baroque theatrical photography, sharp foreground and readable background architecture"
     },
     {
       "id": "color-binding",
-      "category": "color_binding",
-      "prompt": "A blue cube beside a yellow sphere and a green pyramid on a gray table"
+      "title": "04 Dense color and object binding",
+      "category": "dense_color_binding",
+      "prompt": "An elaborate surreal fashion tableau with three models: the left model wears a cobalt-blue coat and holds a yellow glass sphere, the center model wears a saffron dress and holds a green ceramic pyramid, the right model wears an emerald suit and holds a red velvet cube; rococo greenhouse, rare orchids, patterned tile floor, prismatic sunlight, magazine-cover precision, preserve every color-object pairing"
     },
     {
       "id": "spatial-relationship",
-      "category": "spatial_relationship",
-      "prompt": "A blue cube to the left of a yellow sphere, both centered in the frame"
+      "title": "05 Nested spatial relationships",
+      "category": "nested_spatial_relationships",
+      "prompt": "A meticulous cutaway diorama of a vertical city: a glass greenhouse sits directly above a silver subway car, a red fox stands inside the greenhouse beneath a hanging moon lamp, a violinist waits below the subway platform, and a yellow airship passes behind the entire structure; isometric perspective, architectural-section drawing mixed with photoreal materials, dozens of tiny rooms, stairs and people, clean depth"
     },
     {
       "id": "long-prompt",
-      "category": "long_prompt",
-      "prompt": "A precise editorial photo of a compact robotics workbench with labeled tools, a small oscilloscope, coiled cables, a half-assembled drone, and a warm desk lamp; the scene should remain readable without clutter"
+      "title": "06 Cinematic night-market panorama",
+      "category": "cinematic_panorama",
+      "prompt": "A sweeping cinematic panorama of a rain-soaked floating night market at blue hour: in the foreground a chef plates translucent dumplings under a red silk canopy; in the midground children chase paper lanterns across narrow bridges while merchants unload exotic fruit from wooden boats; in the background a terraced megacity rises through mist beneath a storm, with hundreds of warm windows, wet reflections, steam, umbrellas, ropes and signage; deep focus, anamorphic highlights, realistic faces, coherent perspective, painterly color grading with documentary-level detail"
     },
     {
       "id": "english-text-rendering",
-      "category": "text_rendering",
-      "prompt": "A clean street sign with the exact text \"ORBIT QUANT\""
+      "title": "07 Editorial Latin typography",
+      "category": "latin_typography",
+      "prompt": "A sophisticated Swiss International Style exhibition poster photographed behind slightly reflective museum glass, with the exact large headline \"ORBIT QUANT\" and the exact smaller subtitle \"DATA WITHOUT CALIBRATION\"; strict modular grid, red, black and white screenprint, tiny registration marks, embossed paper fibers, sharp letterforms, dramatic gallery shadows, no other text"
     },
     {
       "id": "cyrillic-text-rendering",
-      "category": "text_rendering",
-      "prompt": "A handwritten note in Cyrillic that says \"\u041a\u0412\u0410\u041d\u0422\u041e\u0412\u0410\u041d\u0418\u0415\""
+      "title": "08 Russian Constructivist typography",
+      "category": "russian_typography",
+      "prompt": "A richly detailed Russian Constructivist science-fiction poster with the exact Cyrillic headline \"КВАНТОВАЯ ОРБИТА\", the exact subtitle \"МОСКВА 2049\", and a small exact stamp \"КВАНТОВАНИЕ\"; diagonal red and black geometry, cream paper, cosmonaut portrait, orbital diagrams, halftone grain, folded corners, layered ink, museum archival photograph, all letters crisp and correctly ordered"
     },
     {
       "id": "style-heavy",
-      "category": "style",
-      "prompt": "A crowded train platform at sunset, documentary photo style, high dynamic range, natural grain, realistic faces, detailed architecture"
+      "title": "09 Japanese typography and mixed style",
+      "category": "japanese_typography",
+      "prompt": "An elaborate Japanese art magazine cover combining Edo woodblock printing with a futuristic Tokyo skyline, with the exact vertical title \"量子の軌道\" and the exact subtitle \"東京の未来\"; giant indigo waves curl around glass towers, red-crowned cranes cross a gold moon, tiny pedestrians and trains fill the lower streets, visible washi fibers, layered spot colors, precise Japanese glyphs, balanced editorial layout"
     },
     {
       "id": "occlusion-reflection",
-      "category": "occlusion_reflection",
-      "prompt": "A reflective chrome teapot partly hidden behind orange flowers, with the room visible as a coherent reflection on the metal"
+      "title": "10 Chinese typography, reflection, occlusion",
+      "category": "chinese_typography_reflection_occlusion",
+      "prompt": "A luxurious Chinese retro-futurist department-store window at night with the exact gold title \"量子轨道\" and the exact red subtitle \"未来之城\"; a curved chrome robot is partly occluded by peonies and blue-and-white porcelain, the calligraphy and neon street must appear coherently reflected across its body, multiple glass layers, silk textures, passing bicycles, cinematic rain, fine product-photography detail"
     }
   ]
 }