Update comparison matrix and Metal results

README.md

@@ -32,7 +32,7 @@ OrbitQuant paper itself leaves text encoders in BF16.
 ## Install
 
 ```bash
-pip install "orbitquant[kernels]>=0.3.0" "diffusers>=0.39" "transformers>=5.13" accelerate
+pip install "orbitquant[kernels]>=0.3.1" "diffusers>=0.39" "transformers>=5.13" accelerate
 ```
 
 OrbitQuant uses packed low-bit inference by default. CUDA first uses an importable
@@ -127,39 +127,72 @@ results are in `benchmark/summary.json`.
 
 ## Metal Kernel Reference
 
-The packed Metal operator was also measured on an Apple M2 Max with Torch
-2.12.1, FP16 activations, W4 weights, `in_features=768`, and
-`out_features=2304`. The reference materializes the FP16 weight and then calls
-`F.linear` inside each timed iteration.
-
-| Activation rows | Packed Metal | Materialize + `F.linear` | Speedup |
-| ---: | ---: | ---: | ---: |
-| 1 | 0.0508 ms | 0.2181 ms | 4.29x |
-| 3 | 0.0879 ms | 0.2188 ms | 2.49x |
-| 8 | 0.1698 ms | 0.2230 ms | 1.31x |
-| 9 | 0.0368 ms | 0.2264 ms | 6.15x |
-| 16 | 0.0485 ms | 0.2581 ms | 5.33x |
-| 31 | 0.0568 ms | 0.2635 ms | 4.64x |
-
-This is an operator-level reference, not a full FLUX.2 pipeline benchmark. The
-packed path consumes packed indices directly and does not allocate a complete
-floating-point weight matrix.
+The packed Metal operator was measured on an Apple M2 Max with Torch 2.12.1,
+FP16 activations, W4 weights, `in_features=768`, and `out_features=2304`.
+
+| Activation rows | Packed Metal | Resident FP16 `F.linear` | Materialize + `F.linear` | Packed vs materialize |
+| ---: | ---: | ---: | ---: | ---: |
+| 1 | 0.0470 ms | 0.0411 ms | 0.2310 ms | 4.92x |
+| 2 | 0.0439 ms | 0.0422 ms | 0.2371 ms | 5.40x |
+| 3 | 0.0451 ms | 0.0404 ms | 0.2348 ms | 5.20x |
+| 8 | 0.0420 ms | 0.0503 ms | 0.2512 ms | 5.97x |
+| 9 | 0.0446 ms | 0.0501 ms | 0.2422 ms | 5.43x |
+| 16 | 0.0459 ms | 0.0581 ms | 0.2512 ms | 5.47x |
+| 31 | 0.0428 ms | 0.0659 ms | 0.2623 ms | 6.12x |
+
+One-row projections use the SIMD-group packed matvec. Aligned FP16/BF16
+projections with two or more rows use the padded matrix path. The packed
+weight payload, row norms and centroids occupy 25.26% of the materialized FP16
+weight size for this shape.
+
+For a complete 4096-coordinate RPBH block with constants resident on MPS, the
+512-thread fused activation path measured 1.17-1.45x faster than the 256-thread
+path across 1-4096 activation rows. Smaller or multi-block RPBH dimensions
+retain 256 threads.
+
+These are operator-level references, not a full FLUX.2 pipeline benchmark.
+Packed execution consumes low-bit indices directly and does not allocate a
+complete floating-point weight matrix.
 
 ## Visual Comparison
 
-The matrix contains full-resolution output from ten difficult prompts: micro-detail,
-counting, nested architecture, original mixed-media style, abstract materials, English
-fine print, Russian typography, Japanese typography, Chinese typography and a dense
-multi-subject panorama. Columns use identical prompts, settings and seed. The WebP was
-encoded at quality 95.
-
-![BF16, SDNQ and OrbitQuant comparison](assets/image_generation_comparison_matrix.webp)
-
-All three variants remained coherent and detailed. OrbitQuant preserved material detail,
-reflections and dense compositions without blank or noisy outputs. SDNQ reproduced the
-small English specification text more accurately. Exact object counts and fine CJK text
-were unreliable for every variant. This is a subjective paired inspection, not an
-objective quality metric.
+The matrix uses the complete ten-prompt stress pack with full 1024x1024 tiles
+and WebP quality 95. BF16 is the full-precision reference from the controlled
+visual run; the SDNQ and OrbitQuant columns use the corresponding L40S
+benchmark outputs. Every row uses the same prompt, seed, resolution, step count
+and guidance.
+
+![BF16, SDNQ UINT4 and OrbitQuant W4A4 across ten difficult prompts](assets/image_generation_comparison_matrix.webp)
+
+### Visual Assessment
+
+- **No collapse:** all thirty outputs are finite, coherent and detailed. OrbitQuant did not
+  produce blank, noisy or structurally broken images.
+- **Micro-detail and materials:** all three variants preserve gears, filigree, architectural
+  interiors, paper grain, metal, resin and reflected surfaces. OrbitQuant remains competitive
+  with BF16 and SDNQ in these cases.
+- **Dense composition:** all variants retain foreground/background separation and the main
+  hierarchy in the architectural cutaway and orbital-banquet prompts. Individual requested
+  objects move or disappear because quantization changes the denoising trajectory.
+- **Counting:** none of the variants reliably renders exactly nine performers or every exact
+  repeated motif. This is a base-model limitation in the tested setting rather than an
+  OrbitQuant-only collapse.
+- **English typography:** OrbitQuant is strongest on this row: it preserves the headline,
+  subtitle and all four specification lines. SDNQ preserves the headline and three table
+  lines but omits or corrupts some requested text.
+- **Russian typography:** all variants render the large headline, subtitle and archive stamp
+  well; small contents text contains errors in every column.
+- **Japanese and Chinese typography:** visual glyph quality is plausible, but exact requested
+  strings are not reliably reproduced by any variant.
+- **Trajectory fidelity:** both quantizers change the denoising trajectory at the same seed;
+  neither remains consistently closer to BF16 across all ten prompts.
+
+This subjective paired inspection demonstrates non-collapse and exposes
+concrete failure modes; it is not a substitute for GenEval or another
+task-specific objective metric. In this controlled FLUX.2 Klein 9B comparison,
+OrbitQuant produces the smaller complete 4-bit pipeline, reaches SDNQ
+hot-generation parity with lower runtime memory, and produces the strongest
+English fine-print result in the prompt pack.
 
 ## Limitations