| # PERFORMANCE BENCHMARK PLAN: JiRack 236B | |
| **Document ID:** CMS-JR-236B-BCH-2025 | |
| **Target:** 16x H100 GPU Cluster (2-Node HGX) | |
| --- | |
| ## 1. Stage I: Fabric & Communication (The "Stress Test") | |
| Before loading weights, the inter-node fabric must be verified. JiRack's **14:1 GQA** relies on ultra-fast All-Reduce operations during the attention heads' merge phase. | |
| | **Test Type** | **Tool** | **Target Metric** | **Success Criteria** | | |
| |-------------------------|---------------------------|-------------------------------------------|-----------------------------------------------| | |
| | NCCL All-Reduce | `nccl-tests` | Bus Bandwidth | > 380 GB/s (Intra-node) / > 45 GB/s (Inter-node) | | |
| | P2P Latency | `p2pBandwidthLatencyTest` | Latency (μs) | < 2.0 μs via NVLink | | |
| | IB Write BW | `ib_write_bw` | Throughput | 390+ Gbps per link (NDR 400) | | |
| --- | |
| ## 2. Stage II: JiRack SWA Kernel Fusion (The "Compute Test") | |
| Standard benchmarks (like HPL) are ineffective for JiRack. The **SwiGLU-Attention (SWA)** fusion logic must be tested on the specific **14,336-width dimension**. | |
| - **Benchmark Tool:** `trtllm-bench` (TensorRT-LLM) or a custom Triton kernel profiler. | |
| - **Target Configuration:** | |
| - **Input:** 1024 tokens (Prompt). | |
| - **Output:** 128 tokens (Generation). | |
| - **Batch Size:** 1, 8, 32, 64. | |
| ### **The "Grabko Metric":** | |
| - The system must achieve **at least 55% MFU (Model FLOPs Utilization)**. | |
| - If the vendor delivers **<45%**, BRE pre-fetching logic may be throttled due to PCIe bottlenecks. | |
| --- | |
| ## 3. Stage III: Throughput & Latency (The "User Experience") | |
| This stage simulates real-world commercial usage of the JiRack API. | |
| | **Scenario** | **Metric** | **Target for 236B** | | |
| |-------------------------|----------------------------|-------------------------------------------| | |
| | Interactive (BS=1) | Time to First Token (TTFT) | < 150ms | | |
| | Interactive (BS=1) | Tokens per Second (TPS) | > 25 tokens/sec | | |
| | High Load (BS=64) | Total Throughput | > 1,200 tokens/sec | | |
| --- | |
| ## 4. Stability & Reliability (The "Burn-in") | |
| Large clusters often fail during long reasoning chains. | |
| - **Test:** 24-hour continuous generation at 80% TDP (Thermal Design Power). | |
| - **Success Criteria:** | |
| - Zero NCCL timeouts. | |
| - Zero XID errors (GPU hardware faults). | |
| ### **MTBF Target:** | |
| - Mean Time Between Failures (MTBF) for the 108-layer stack must exceed **720 hours** on this specific 16-GPU setup. | |
| --- | |
| ## 5. Official Verification | |
| The vendor must provide a **logs export** containing the `proof_of_authorship` string verification. | |
| ```python | |
| verify_authorship(model) -> "Author: Konstantin Vladimirovich Grabko (CMS Manhattan) 2025" | |
| ``` | |
| Acceptance of the hardware cluster is contingent upon meeting these 236B benchmarks. |