Upload README.md with huggingface_hub

README.md

@@ -21,20 +21,26 @@ host RAM as int8, dequantized to fp16 on the fly). It reads only the **2 active
 16), so it's faster than running them dense. Quant: **int8 experts + fp16 attention / router / lm-head**
 (int16 activations break attention; W4 is too coarse). Fits the device in **4 QNN contexts** (the HTP caps
 concurrent contexts ≈ 8). The Phi-3 tokenizer + chat template (`<|user|> … <|end|><|assistant|>`) are applied
-on-device, so you pass plain **text**.
+on-device, so you pass plain **text**. **MAXCTX = 2048** (16 Q-heads → wide attention is HTP-correct past the
+v79 512 wall — verified 12/12 vs the fp32 reference on a 529-token needle-recall prompt).
 
 | Metric | Value |
 |---|---|
-| **Decode** | **~5–7.7 tok/s** (top-2 dynamic-weight; thermal-dependent) |
-| **TTFT** | ~140–190 ms |
-| **Accuracy** | **100 % greedy parity** vs the fp32 PyTorch reference (5 prompts) |
-| **Peak RSS** | ~4.8 GB |
+| **Decode** | **~2.1 tok/s** (top-2 dynamic-weight; static 2048-wide attention every token) |
+| **Prefill / TTFT** | **~0.46 s × prompt_length** (decode-over-prompt; ≈4.6 s @10 tok, ≈4 min @529 tok) |
+| **Accuracy** | **100 % greedy parity** vs the fp32 reference — 5 prompts **+ a 529-token long-context test** |
+| **Context** | **2048** tokens |
+| **Peak RSS** | ~4.9 GB |
+
+> Decode speed is set by the compiled MAXCTX (the 2048-wide attention runs every token), so this is the
+> long-context configuration, not the fastest. The two open levers are a **batched prefill graph** (kills the
+> O(L0) prefill) and **GQA-native attention** (the a-graphs currently materialize 4→16 KV heads).
 
 ## Contents (`v79/`)
 | file | what | size |
 |---|---|---|
 | `phimoe.json` | the QHexRT manifest (`phimoe_generate` host-op) | small |
-| `a_lo.bin` / `a_hi.bin` | attn+router graphs, layers 0–15 / 16–31 (KV cache) | 648 MB each |
+| `a_lo.bin` / `a_hi.bin` | attn+router graphs, layers 0–15 / 16–31 (KV cache, MAXCTX 2048) | 655 MB each |
 | `ffndyn.bin` | one dynamic-weight expert-FFN graph (reused) | 72 KB |
 | `lmhead_ar1.bin` | final LayerNorm + lm-head → logits | 251 MB |
 | `experts_i8.bin` | all 512 experts (32×16), per-output-channel int8 | 2.69 GB |
@@ -59,7 +65,9 @@ adb shell "cd /data/local/tmp/phimoe && export ADSP_LIBRARY_PATH='/data/local/tm
 
 ## Caveats
 - **v79 only** (SM8750). Another arch = re-export (the build plane / `npu-forge`).
-- MAXCTX 64 in this build (the AR=1 KV cache width); longer contexts = a re-export.
+- **MAXCTX 2048** (prompt + generation ≤ 2048). The decode-over-prompt prefill is ~0.46 s/prompt-token, so very
+  long prompts take minutes to the first token; decode is ~2.1 tok/s regardless of length. A faster small-window
+  variant (e.g. 512 + sliding KV ring) is a re-export (`MAXCTX=512 build_alo_ahi_2048_v2.sh`).
 - Greedy/argmax decode (temperature 0). Weight-only int8 occasionally flips a thin-margin token; output stays
   coherent (the prior-port finding — int8 is the accuracy/memory sweet spot, W4 too coarse).