Refresh GLM Q2TP and Q4TP performance evidence

README.md

@@ -126,36 +126,50 @@ CMF_GLM_DYNAMIC_MOE=0 cortiq run glm-5.3-flash-q2tp.cmf \
 
 Automatic mode deliberately keeps both profiles on the exact CPU MoE path. On
 the measured host, bounded Vulkan expert pooling was slower for both Q2TP and
-Q4TP. Explicit `CMF_GLM_DYNAMIC_MOE=1` remains available for compatibility and
-experimentation; `CMF_GLM_DYNAMIC_MOE=0` always disables it. The GLM-specific
-pool is limited to 20–40% of the configured physical-memory envelope, and an
-explicit `CMF_GLM_EXPERT_SLOTS` value cannot bypass that cap.
+Q4TP, so it is not selected automatically. Explicit `CMF_GLM_DYNAMIC_MOE=1`
+remains available for compatibility and experimentation; `CMF_GLM_DYNAMIC_MOE=0`
+always disables it. The GLM-specific pool is limited to 20–40% of the configured
+physical-memory envelope, and an explicit `CMF_GLM_EXPERT_SLOTS` value cannot
+bypass that cap.
+
+The final runtime pass also reduces dispatch overhead without changing model
+bytes or arithmetic. Q4TP combines compatible shared-input projection groups
+into one tiled dispatch (only when every member uses Q4TP), while retaining
+independent routed and shared-expert accumulation. Q2TP batches all selected
+expert gate/up rows and the weighted down rows through the existing bounded
+worker path, reusing thread-local scratch. The Q2TP CPU MoE cadence fell from
+about 1,577 serial jobs per token to 527–530; greedy output and the official
+template oracle remained unchanged.
 
 Measured on an RTX 3090 with dual AMD EPYC 7H12 CPUs and 32 CPU worker threads:
 
 | execution path | steady decode | notes |
 |---|---:|---|
-| Q2TP CPU32, automatic | **2.498 tok/s** | 20-token clean run, zero GPU submissions |
-| Q2TP CPU32, explicit CPU | **2.492 tok/s** | 20-token clean run |
-| Q2TP forced dynamic Vulkan, 16-GB target | **1.413 tok/s** | 512 slots / 4,352 MB pool, 42 submissions/token |
-| Q4TP CPU, final run A | 1.912 tok/s | exact path, zero GPU submissions |
-| Q4TP CPU, final run B | 2.350 tok/s | exact path, zero GPU submissions |
-| Q4TP CPU, final run C | 2.207 tok/s | exact path, zero GPU submissions |
-| Q4TP CPU, three-run mean | **2.156 tok/s** | 1.26% below the earlier 2.184 tok/s baseline |
-| Q4TP forced dynamic Vulkan | **0.634 tok/s** | 40-token run, 42 submissions/token, 13.9 GB pool |
-
-The Q4TP CPU result varies with NUMA placement; the three-run spread is reported
-instead of presenting one noisy maximum. Forced Vulkan was about 3.4× slower,
-which is why it is not the Q4TP default.
-
-For Q2TP, a bounded compatibility run with
+| Q2TP CPU32, explicit CPU | **2.955 tok/s** | 20-token clean run, 4,071 allocations/token, 530 pool dispatches/token, zero GPU submissions |
+| Q2TP automatic (warm CPU32) | **3.063 tok/s** | 20-token clean run, exact CPU path, zero GPU submissions/uploads |
+| Q2TP forced dynamic Vulkan, normal budget | 1.240 tok/s | 752 slots / 6,392 MB pool, 42 submissions/token, 7,564 MiB peak |
+| Q2TP forced dynamic Vulkan, 16,000-MB target | 1.302 tok/s | 512 slots / 4,352 MB pool, 42 submissions/token, 6,563 MiB peak |
+| Q4TP CPU32, final runs A/B | **2.685 / 2.648 tok/s** | mean **2.666 tok/s**, 5,248 allocations/token, zero GPU submissions |
+| Q4TP automatic | same exact CPU path | no dynamic pool, zero GPU submissions/uploads |
+| Q4TP forced dynamic Vulkan, normal budget | 1.278 tok/s | 512 slots / 6,400 MB pool, 42 submissions/token, 6,731 MiB peak |
+| Q4TP forced dynamic Vulkan, 16,000-MB target | 1.430 tok/s | 344 slots / 4,300 MB pool, 42 submissions/token, 6,475 MiB peak |
+
+The Q4TP CPU result varies with NUMA placement; the final A/B mean is reported
+instead of presenting one noisy maximum. A warm Q4TP regression run measured
+2.614 tok/s and remained within that accepted spread. Forced Vulkan remains an
+explicit compatibility path: it is slower than exact CPU for both profiles,
+which is why neither profile selects it automatically.
+
+For both profiles, bounded compatibility runs with
 `CMF_GPU_VRAM_MB=16000 CMF_GLM_DYNAMIC_MOE=1` completed successfully with the
-official-template answer `4`. Its 4,352-MB/512-slot expert pool peaked at
-6,563 MiB of total sampled process VRAM on the physical 24-GB test card, safely
-below 16 GiB. The automatic normal-budget run also emitted `4`, used the exact
-CPU path, and peaked at 331 MiB of VRAM. CPU-only systems and smaller adapters
-remain valid through the same exact fallback. Metal will be tested from these
-immutable artifacts.
+official-template answer `4`. The Q2TP 4,352-MB/512-slot pool peaked at 6,563
+MiB, and the Q4TP 4,300-MB/344-slot pool peaked at 6,475 MiB, both safely below
+the simulated 16,000-MB target. These samples were taken on the physical
+24,576-MiB (24-GB) RTX 3090; the 16,000-MB value is an explicit software
+budget, not the card's capacity. The automatic Q2TP normal-budget run also
+emitted `4`, stayed on exact CPU, and peaked at 331 MiB of VRAM. CPU-only
+systems and smaller adapters remain valid through the same exact fallback.
+Metal will be tested from these immutable artifacts.
 
 ## Scope and license