sm280299 commited on
Commit
01d77ac
·
verified ·
1 Parent(s): 043d597

Scrub internal engineering notes from manifest _comment

Browse files
Files changed (1) hide show
  1. v79/phimoe.json +1 -1
v79/phimoe.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "schema_version": 1,
3
- "_comment": "Phi-tiny-MoE (microsoft/Phi-tiny-MoE-instruct) FIRST MoE on v79, native QHexRT LLM via the phimoe_generate host-op. @ MAXCTX 2048. DECODE: per-layer GQA-NATIVE attn+router graphs a0..a31 (K/V at 4 heads, no repeat_interleave/spill; 16 Q-heads -> wide attn HTP-correct >512) + host sparsemixer top-2 + ONE FUSED 2-expert FFN (ffn2) + in-graph lm-head. BATCHED PREFILL: pf_lo/pf_hi (32 causal attn+router graphs over the whole prompt [PN=576] in ONE forward/layer) + ffn_pf (batched single-expert FFN; each used expert run once/layer, expert-grouped) seed the decode KV cache, then AR=1 decode — collapses the O(L0) decode-over-prompt. Quant = int8 experts + fp16 attn/router/lm-head. 7 QNN contexts (4 decode + 3 prefill) <= HTP ~8 limit. Prompt: text (Phi-3 chat template) OR raw ids; prompts >PN fall back to decode-over-prompt. PERF (verified 100% greedy parity incl. a 529-token needle-recall prompt): decode ~5-7 tok/s (best ~7); prefill TTFT ~2.8 s for a 529-token prompt (~34x over decode-over-prompt's 95 s). GQA-native + fused-FFN gave ~2.5-3.3x decode; batched prefill gave ~34x TTFT.",
4
  "model": { "name": "phi-tiny-moe", "family": "llm", "dsp_arch": "v79", "tokenizer_pre": "default" },
5
  "params": {
6
  "hidden": 4096, "vocab": 32064, "n_layers": 32, "max_ctx": 2048,
 
1
  {
2
  "schema_version": 1,
3
+ "_comment": "phi-tiny-moe - Qualcomm Hexagon v79 (QHexRT) deployment manifest.",
4
  "model": { "name": "phi-tiny-moe", "family": "llm", "dsp_arch": "v79", "tokenizer_pre": "default" },
5
  "params": {
6
  "hidden": 4096, "vocab": 32064, "n_layers": 32, "max_ctx": 2048,