Hot/cold expert analysis: skew, profile pinning hit rates; fix hit counting and default pin fraction

README.md

@@ -42,7 +42,7 @@ System information recorded on 2026-10-23:
 | Container swap limit | 2,936,848,384 bytes (about 2.74 GiB); measured test processes used 0 swap |
 | GPU | No GPU used; CPU-only build and inference |
 | OS / kernel | Amazon Linux 2023 x86_64, Linux 6.18.48 |
-| llama.cpp | `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` (`970b51c`) + `flint-epq.patch` (sha256 `b8a4e77af3743bd35bd4c14258c70c12031e60ba264f95ab93593519ea0ab08c`) |
+| llama.cpp | `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` (`970b51c`) + `flint-epq.patch` (sha256 `94559012e2ea65cad6ce2175927c19095d13256bb4ec8fb2fdcd163570e00385`) |
 
 ## Build
 
@@ -170,6 +170,35 @@ Separate workload from the memory tiers: context 512, batch 128, ubatch 32, 32 g
 
 v2 generates about 2.9× faster than v1 and about 75 % of native generation speed in this workload. The prompt phase is also faster in v2, likely because the preload removes SSD reads during prefill; this was not measured separately.
 
+## Hot and cold experts
+
+Routing is not uniform. Flint's experts differ widely in how often they are used, so a cache that keeps frequently used ("hot") experts resident and streams the rest should help in principle. Measured on this checkpoint:
+
+- **Skew is real.** Per layer, the Gini coefficient of expert request counts is 0.38–0.62 (0 = uniform). The top 6 % of experts take 19–36 % of requests (uniform would be 6 %). Across two different texts, per-layer rank correlation of expert counts has median 0.58.
+- **A profile transfers across texts moderately.** Experts chosen as the top 10 % per layer from a profiling text cover 28 % of requests in a held-out text (uniform: 10 %).
+- **The working set is much larger than small arenas.** One token uses 47 layers × 6 = 282 distinct (layer, expert) pairs, and an expert is typically not reused until the next token. A cache of fewer than about 282 slots (≈ 0.7 GiB) gets almost no hits under LRU, because the entries are evicted before they are reused.
+- **Pinning a profiled hot set helps only in a narrow band.** Real runs (514-token held-out text, 2 chunks of 256 tokens per setting, `-ub 1`):
+
+| Arena | LRU hit rate | Profile-pinned 25 % hit rate | Pinned − LRU |
+|---:|---:|---:|---:|
+| 128 MiB (52 slots) | 0 % | 0 % | 0 |
+| 256 MiB (105 slots) | 0 % | 0 % | 0 |
+| 1 GiB (420 slots) | 19.3 % | 21.5 % | +2.2 points |
+| 3 GiB (1,260 slots) | 40.3 % | 37.5 % | −2.8 points |
+
+Perplexity was identical (6.4793) for every LRU and pinned run, so caching changes speed only, not results.
+
+Two bugs in the existing hot/cold code were found and fixed in this patch:
+
+1. **Hit requests were not counted in the profile.** The counter was only incremented on the miss path, so experts that are hot because they hit were underestimated. Now every request is counted.
+2. **Without `SDQ_HOT_PIN`, the loader pinned every expert that appeared in the profile**, which left no cold slot to evict. The default is now 25 % of the arena slots. Pinning still needs an explicit profile (`SDQ_PROFILE_IN`); without one, the cache is plain LRU.
+
+Simulation (`results/hot/sim.py`, trace replay with uniform slots, 1,017 held-out tokens) gives a somewhat larger pinning gain at 1 GiB (+3.2 points) and a small gain at 3 GiB (+0.7). It disagrees with the real pager below 256 MiB, where it predicts 1–2 % hits and the pager gives 0 %.
+
+**Conclusion.** The hot/cold effect is real but modest on this checkpoint, and it changes sign with arena size. The limit that matters most is the per-token working set: below about 0.7 GiB, each token re-reads most of its experts from SSD regardless of the policy. That is why the 2–4 GiB tiers are slow. Pinning stays opt-in, and the defaults are unchanged.
+
+Files: `results/hot/profile.bin` (the profile used), `results/hot/analyze.py`, `results/hot/sim.py`, `results/hot/stats_*.jsonl` (hit rates), `results/hot/test_*.log`, `results/hot/prof_build.log`.
+
 ## Version coverage
 
 | Feature | Version | Status for Flint |

flint-epq.patch

@@ -1666,10 +1666,10 @@ index 0000000..c1bbe6e
 +} // namespace sdq
 diff --git a/src/sdq_pager.cpp b/src/sdq_pager.cpp
 new file mode 100644
-index 0000000..11c9331
+index 0000000..d837a14
 --- /dev/null
 +++ b/src/sdq_pager.cpp
-@@ -0,0 +1,2499 @@
+@@ -0,0 +1,2505 @@
 +// SDQ expert pager 實作：見 sdq_pager.h 的設計說明
 +#include "sdq_pager.h"
 +
@@ -3223,7 +3223,10 @@ index 0000000..11c9331
 +              [](const auto & a, const auto & b) { return a.first > b.first; });
 +
 +    hot_.assign((size_t) info_.n_layer * (size_t) info_.n_expert, 0);
-+    const int want = cfg_.hot_pin > 0 ? cfg_.hot_pin : (int) ranked.size();
++    // 預設釘選 25 % 的槽位（模擬顯示這是 1 GiB 階層的最佳點）。舊版沒有設定時
++    // 會把所有出現過的 expert 都標成熱門，等於沒有冷槽可淘汰。
++    const int want = cfg_.hot_pin > 0 ? cfg_.hot_pin
++                                      : std::max<int>(1, (int) (n_slots_ / 4));
 +    n_hot_ = 0;
 +    for (const auto & r : ranked) {
 +        if (n_hot_ >= want) {
@@ -3409,6 +3412,9 @@ index 0000000..11c9331
 +        slots_[s].last_use = ++clock_;
 +        slots_[s].pin++;
 +        touch_freq_locked(slots_[(size_t) s]);
++        // 剖面必須統計「所有」請求（命中也要算）。舊版只在 miss 路徑計數，
++        // 熱門 expert 因為大多命中，會被剖面系統性低估。
++        count_request(il, ie);
 +        stats_.expert_requests.fetch_add(1, std::memory_order_relaxed);
 +        stats_.expert_hits.fetch_add(1, std::memory_order_relaxed);
 +        if (lstats_ && il >= 0 && il < (int) info_.n_layer) {

results/hot/analyze.py

@@ -0,0 +1,49 @@
+import numpy as np, collections, sys
+def load(path):
+    trace=[]  # list of (layer, [experts per token]) in order
+    for line in open(path):
+        p=line.split()
+        il=int(p[0]); nt=int(p[1])
+        toks=[]
+        for tok in p[2:2+nt]:
+            e=tok.split(":")[1]
+            toks.append([int(x) for x in e.split(",")])
+        trace.append((il,toks))
+    return trace
+prof=load("route_profile.txt"); test=load("route_test.txt")
+NL,NE=49,256
+def counts(tr):
+    c=np.zeros((NL,NE),np.int64)
+    for il,toks in tr:
+        for t in toks:
+            for e in t: c[il,e]+=1
+    return c
+cp=counts(prof); ct=counts(test)
+layers=[l for l in range(NL) if cp[l].sum()>0]
+print("layers with routing:",len(layers), " requests/layer:", cp[layers[0]].sum())
+def gini(x):
+    x=np.sort(x.astype(float)); n=len(x); 
+    return (2*np.arange(1,n+1)-n-1).dot(x)/(n*x.sum())
+def top_share(x,k):
+    s=np.sort(x)[::-1]; return s[:k].sum()/s.sum()
+print("layer  gini   top6%  top25%  max/mean  unused  (profile corpus)")
+for l in layers[:6]+layers[-3:]:
+    x=cp[l]; print(f"{l:5d}  {gini(x):.3f}  {top_share(x,int(0.06*NE)):.3f}  {top_share(x,int(0.25*NE)):.3f}  {x.max()/x.mean():6.2f}  {(x==0).sum():4d}")
+allc=cp[layers].sum(0)
+print("ALL layers pooled gini %.3f"%gini(cp[layers].ravel()))
+# stability: rank correlation of per-(layer,expert) counts between corpora
+
+def rank(x):
+    o=np.argsort(x,kind="mergesort"); r=np.empty(len(x)); r[o]=np.arange(len(x)); return r
+r=np.corrcoef(rank(cp[layers].ravel()),rank(ct[layers].ravel()))[0,1]
+print("spearman(profile counts, test counts) over all (layer,expert): %.3f"%r)
+# coverage: test requests covered by profile top-K per layer
+for frac in [0.05,0.1,0.25,0.5]:
+    K=int(frac*NE); cov=0; tot=0
+    for l in layers:
+        top=set(np.argsort(-cp[l])[:K].tolist())
+        for e in range(NE):
+            if e in top: cov+=ct[l,e]
+        tot+=ct[l].sum()
+    print(f"top {int(frac*100)}% of experts per layer (from profile) covers {100*cov/tot:.1f}% of test requests")
+np.save("cp.npy",cp); np.save("ct.npy",ct)

results/hot/pin40_256_chunk1.log

@@ -0,0 +1,30 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq] expert 剖面：/tmp/opencode/flint-work/hot/profile.bin → 釘選 40 / 11817 個熱門 expert（永不淘汰）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：40 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.30 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.00.620.112 I cmn          init: llama threadpool init, n_threads = 16
+0.01.110.033 I 
+0.01.110.139 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.110.162 I perplexity: tokenizing the input ..
+0.01.116.123 I perplexity: tokenization took 5.956 ms
+0.01.116.144 I perplexity: calculating perplexity over 1 chunks, n_ctx=256, batch_size=256, n_seq=1
+1.06.492.133 I perplexity: 65.38 seconds per pass - ETA 1.08 minutes
+[1]9.4981,
+1.06.504.932 I Final estimate: PPL = 9.4981 +/- 2.88052
+
+[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 11997 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：40 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.36 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/pin80_256_chunk1.log

@@ -0,0 +1,30 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq] expert 剖面：/tmp/opencode/flint-work/hot/profile.bin → 釘選 80 / 11817 個熱門 expert（永不淘汰）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：80 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.30 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.00.610.354 I cmn          init: llama threadpool init, n_threads = 16
+0.01.042.629 I 
+0.01.042.716 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.042.733 I perplexity: tokenizing the input ..
+0.01.048.348 I perplexity: tokenization took 5.612 ms
+0.01.048.362 I perplexity: calculating perplexity over 1 chunks, n_ctx=256, batch_size=256, n_seq=1
+1.01.697.597 I perplexity: 60.65 seconds per pass - ETA 1.00 minutes
+[1]9.4981,
+1.01.708.101 I Final estimate: PPL = 9.4981 +/- 2.88052
+
+[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 11997 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：80 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.36 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/prof_build.log

@@ -0,0 +1,36 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=420  arena=1.00 GiB  (RAM 實測 RSS=1.05 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 420
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.01.057.954 I cmn          init: llama threadpool init, n_threads = 16
+0.01.555.826 I 
+0.01.555.935 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.555.955 I perplexity: tokenizing the input ..
+0.01.564.703 I perplexity: tokenization took 8.742 ms
+0.01.564.721 I perplexity: calculating perplexity over 4 chunks, n_ctx=256, batch_size=256, n_seq=1
+0.53.731.585 I perplexity: 52.17 seconds per pass - ETA 3.47 minutes
+[1]13.7740,[2]14.8027,[3]9.1885,[4]10.8376,
+3.49.646.713 I Final estimate: PPL = 10.8376 +/- 1.61503
+
+[sdq] MoE op 路徑計數：row-parallel 9 次，mode C 47700 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=420  arena=1.00 GiB  (RAM 實測 RSS=1.12 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 420
+[sdq]   prefetch=off  fadvise_dontneed=1
+[sdq] expert 剖面已寫到 /tmp/opencode/flint-work/hot/profile.bin（11817 個有被用到，總請求 286254）
+[sdq]   前   512 名涵蓋  24.41% 的請求
+[sdq]   前  1024 名涵蓋  36.74% 的請求
+[sdq]   前  1536 名涵蓋  45.96% 的請求
+[sdq]   前  2048 名涵蓋  53.45% 的請求
+[sdq]   前  3072 名涵蓋  65.14% 的請求
+[sdq]   前  4096 名涵蓋  73.93% 的請求

results/hot/profile.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b84d2a66d61cd58a05bf65bc4178fe28de72a62f59b12ba6a2b60c8f2e78d911
+size 141820

results/hot/sim.py

@@ -0,0 +1,89 @@
+import numpy as np, collections
+def load(path):
+    out=[]
+    for line in open(path):
+        p=line.split(); il=int(p[0]); nt=int(p[1])
+        for tok in p[2:2+nt]:
+            out.append((il,[int(x) for x in tok.split(":")[1].split(",")]))
+    return out            # list of (layer, experts) in execution order
+prof=load("route_profile.txt"); test=load("route_test.txt")
+NL,NE=49,256
+def counts(tr):
+    c=np.zeros((NL,NE),np.int64)
+    for il,ex in tr:
+        for e in ex: c[il,e]+=1
+    return c
+cp=counts(prof); ct=counts(test)
+def simulate(trace, S, policy, pinned=frozenset()):
+    """global cache of S slots over (layer,expert). pinned keys never evicted (until table full)."""
+    cache=collections.OrderedDict()   # LRU order
+    freq=collections.Counter()
+    hits=miss=0
+    for il,ex in trace:
+        for e in ex:
+            key=(il,e); freq[key]+=1
+            if key in cache:
+                hits+=1
+                cache.move_to_end(key)
+                continue
+            miss+=1
+            if len(cache)>=S:
+                if policy=="lru" or not pinned:
+                    cache.popitem(last=False)
+                else:
+                    # evict least-recent non-pinned; pinned only if all are pinned
+                    victim=None
+                    for k in cache:
+                        if k not in pinned: victim=k; break
+                    if victim is None: cache.popitem(last=False)
+                    else: del cache[victim]
+            cache[key]=True
+    return hits/(hits+miss)
+def lfu_run(trace,S,prior):
+    # LFU with a fixed prior count table; evict min(freq) among cached
+    cache={}; freq=collections.Counter(); hits=miss=0
+    for il,ex in trace:
+        for e in ex:
+            key=(il,e); freq[key]+=1
+            if key in cache: hits+=1; continue
+            miss+=1
+            if len(cache)>=S:
+                v=min(cache, key=lambda k:(freq[k]+prior[k],))
+                del cache[v]
+            cache[key]=True
+    return hits/(hits+miss)
+def top_keys(c, H):
+    flat=np.argsort(-c.ravel())[:H]
+    return frozenset((int(i//NE), int(i%NE)) for i in flat)
+print("arena slots (MiB)  | LRU   | static-hot profile 25%/50% of slots | oracle(test) 50% | LFU(profile prior)")
+for S,label in [(52,"128MiB"),(105,"256MiB"),(420,"1GiB"),(1260,"3GiB"),(4201,"10GiB"),(7561,"18GiB")]:
+    lru=simulate(test,S,"lru")
+    row=[f"{lru*100:5.1f}"]
+    for frac in [0.25,0.5]:
+        H=int(S*frac)
+        pin=top_keys(cp,H)
+        row.append(f"{simulate(test,S,'pin',pin)*100:5.1f}")
+    orc=simulate(test,S,'pin',top_keys(ct,int(S*0.5)))
+    prior=collections.Counter({k:v for k,v in zip(
+        [(int(i//NE),int(i%NE)) for i in range(NL*NE)], cp.ravel())})
+    lf=lfu_run(test,S,prior)
+    print(f"{S:6d} ({label:>7}) | {row[0]} | {row[1]} / {row[2]} | {orc*100:5.1f} | {lf*100:5.1f}")
+
+def per_layer_lru(trace, S):
+    per=S//NL
+    caches={l:collections.OrderedDict() for l in range(NL)}
+    hits=miss=0
+    for il,ex in trace:
+        c=caches[il]
+        for e in ex:
+            if e in c: hits+=1; c.move_to_end(e); continue
+            miss+=1
+            if per>0:
+                if len(c)>=per: c.popitem(last=False)
+                c[e]=True
+    return hits/(hits+miss)
+print("\nper-layer LRU partition (S/47 per layer) vs global LRU vs pin25:")
+for S,label in [(52,"128MiB"),(105,"256MiB"),(420,"1GiB"),(1260,"3GiB")]:
+    pl=per_layer_lru(test,S); g=simulate(test,S,"lru"); p=simulate(test,S,'pin',top_keys(cp,int(S*0.25)))
+    print(f"{S:6d} ({label:>7}) per-layer LRU {pl*100:5.1f} | global LRU {g*100:5.1f} | pin25 {p*100:5.1f}")
+print("\nper-token working set = %d distinct (layer,expert) pairs"%(NL*6))

results/hot/stats_lru_1024.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":23,"expert_misses":535,"hit_rate":4.12186,"ssb_demand_bytes":1022918656,"ssb_prefetch_bytes":0,"ssb_total_bytes":1022918656,"ssd_bytes_per_token":5.11459e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":115,"demand_wait_us":1328610,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":2894528512,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":28114944,"prefetch_bytes":0},{"layer":5,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":10,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":11,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":12,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":13,"req":12,"hit":3,"miss":9,"hit_rate":25,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":16,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":17,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":18,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":19,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":45,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":46,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":13270,"expert_misses":58712,"hit_rate":18.4352,"ssb_demand_bytes":112837246976,"ssb_prefetch_bytes":0,"ssb_total_bytes":112837246976,"ssd_bytes_per_token":4.37354e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":58292,"demand_wait_us":138753272,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":2982920192,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":37,"miss":1511,"hit_rate":2.39018,"demand_bytes":2450866176,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":52,"miss":1496,"hit_rate":3.35917,"demand_bytes":3235381248,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":129,"miss":1419,"hit_rate":8.33333,"demand_bytes":3626827776,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":185,"miss":1363,"hit_rate":11.9509,"demand_bytes":2947743744,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":150,"miss":1398,"hit_rate":9.68992,"demand_bytes":3023437824,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":211,"miss":1337,"hit_rate":13.6305,"demand_bytes":2891513856,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":296,"miss":1252,"hit_rate":19.1214,"demand_bytes":2882043904,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":237,"miss":1311,"hit_rate":15.3101,"demand_bytes":2835283968,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":205,"miss":1343,"hit_rate":13.2429,"demand_bytes":2904489984,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":260,"miss":1288,"hit_rate":16.7959,"demand_bytes":2785542144,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":298,"miss":1250,"hit_rate":19.2506,"demand_bytes":2703360000,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":276,"miss":1272,"hit_rate":17.8295,"demand_bytes":2750939136,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":361,"miss":1187,"hit_rate":23.3204,"demand_bytes":2567110656,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":330,"miss":1218,"hit_rate":21.3178,"demand_bytes":2634153984,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":391,"miss":1157,"hit_rate":25.2584,"demand_bytes":2502230016,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":338,"miss":1210,"hit_rate":21.8346,"demand_bytes":2616852480,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":333,"miss":1215,"hit_rate":21.5116,"demand_bytes":2627665920,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":354,"miss":1194,"hit_rate":22.8682,"demand_bytes":1995374592,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":312,"miss":1236,"hit_rate":20.155,"demand_bytes":2268069888,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":340,"miss":1208,"hit_rate":21.9638,"demand_bytes":2295857152,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":286,"miss":1262,"hit_rate":18.4755,"demand_bytes":2398486528,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":358,"miss":1190,"hit_rate":23.1266,"demand_bytes":2261647360,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":354,"miss":1194,"hit_rate":22.8682,"demand_bytes":2190999552,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":234,"miss":1314,"hit_rate":15.1163,"demand_bytes":2411200512,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":272,"miss":1276,"hit_rate":17.5711,"demand_bytes":2341470208,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":238,"miss":1310,"hit_rate":15.3747,"demand_bytes":2833121280,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":296,"miss":1252,"hit_rate":19.1214,"demand_bytes":2092302336,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":414,"miss":1134,"hit_rate":26.7442,"demand_bytes":1895104512,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":325,"miss":1223,"hit_rate":20.9948,"demand_bytes":2043838464,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":274,"miss":1274,"hit_rate":17.7003,"demand_bytes":2129068032,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":361,"miss":1187,"hit_rate":23.3204,"demand_bytes":2567110656,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":375,"miss":1173,"hit_rate":24.2248,"demand_bytes":2152464384,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":333,"miss":1215,"hit_rate":21.5116,"demand_bytes":2229534720,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":273,"miss":1275,"hit_rate":17.6357,"demand_bytes":2130739200,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":297,"miss":1251,"hit_rate":19.186,"demand_bytes":2090631168,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":352,"miss":1196,"hit_rate":22.739,"demand_bytes":2194669568,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":337,"miss":1211,"hit_rate":21.77,"demand_bytes":2023784448,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":314,"miss":1234,"hit_rate":20.2842,"demand_bytes":2264399872,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":334,"miss":1214,"hit_rate":21.5762,"demand_bytes":2028797952,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":369,"miss":1179,"hit_rate":23.8372,"demand_bytes":1970307072,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":284,"miss":1264,"hit_rate":18.3463,"demand_bytes":2112356352,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":283,"miss":1265,"hit_rate":18.2817,"demand_bytes":2114027520,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":231,"miss":1317,"hit_rate":14.9225,"demand_bytes":2200928256,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":237,"miss":1311,"hit_rate":15.3101,"demand_bytes":2190901248,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":324,"miss":1224,"hit_rate":20.9302,"demand_bytes":2246049792,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":257,"miss":1291,"hit_rate":16.6021,"demand_bytes":2157477888,"prefetch_bytes":0},{"layer":48,"req":774,"hit":163,"miss":611,"hit_rate":21.0594,"demand_bytes":1021083648,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":25819,"expert_misses":117587,"hit_rate":18.0041,"ssb_demand_bytes":225806499840,"ssb_prefetch_bytes":0,"ssb_total_bytes":225806499840,"ssd_bytes_per_token":4.39312e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":117167,"demand_wait_us":282251352,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":2984886272,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":77,"miss":3007,"hit_rate":2.49676,"demand_bytes":4877402112,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":117,"miss":2967,"hit_rate":3.79377,"demand_bytes":6416695296,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":278,"miss":2806,"hit_rate":9.01427,"demand_bytes":7171866624,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":294,"miss":2790,"hit_rate":9.53307,"demand_bytes":6033899520,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":314,"miss":2770,"hit_rate":10.1816,"demand_bytes":5990645760,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":425,"miss":2659,"hit_rate":13.7808,"demand_bytes":5750587392,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":616,"miss":2468,"hit_rate":19.9741,"demand_bytes":5681217536,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":551,"miss":2533,"hit_rate":17.8664,"demand_bytes":5478088704,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":420,"miss":2664,"hit_rate":13.6187,"demand_bytes":5761400832,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":541,"miss":2543,"hit_rate":17.5422,"demand_bytes":5499715584,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":594,"miss":2490,"hit_rate":19.2607,"demand_bytes":5385093120,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":568,"miss":2516,"hit_rate":18.4176,"demand_bytes":5441323008,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":693,"miss":2391,"hit_rate":22.4708,"demand_bytes":5170987008,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":672,"miss":2412,"hit_rate":21.7899,"demand_bytes":5216403456,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":747,"miss":2337,"hit_rate":24.2218,"demand_bytes":5054201856,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":677,"miss":2407,"hit_rate":21.952,"demand_bytes":5205590016,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":686,"miss":2398,"hit_rate":22.2438,"demand_bytes":5186125824,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":694,"miss":2390,"hit_rate":22.5032,"demand_bytes":3994091520,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":714,"miss":2370,"hit_rate":23.1518,"demand_bytes":4348968960,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":654,"miss":2430,"hit_rate":21.2062,"demand_bytes":4618321920,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":619,"miss":2465,"hit_rate":20.0713,"demand_bytes":4684840960,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":664,"miss":2420,"hit_rate":21.5305,"demand_bytes":4599316480,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":718,"miss":2366,"hit_rate":23.2815,"demand_bytes":4341628928,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":490,"miss":2594,"hit_rate":15.8885,"demand_bytes":4760010752,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":538,"miss":2546,"hit_rate":17.4449,"demand_bytes":4671930368,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":456,"miss":2628,"hit_rate":14.786,"demand_bytes":5683544064,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":539,"miss":2545,"hit_rate":17.4773,"demand_bytes":4253122560,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":753,"miss":2331,"hit_rate":24.4163,"demand_bytes":3895492608,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":575,"miss":2509,"hit_rate":18.6446,"demand_bytes":4192960512,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":550,"miss":2534,"hit_rate":17.834,"demand_bytes":4234739712,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":706,"miss":2378,"hit_rate":22.8923,"demand_bytes":5142872064,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":678,"miss":2406,"hit_rate":21.9844,"demand_bytes":4415029248,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":629,"miss":2455,"hit_rate":20.3956,"demand_bytes":4504944640,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":514,"miss":2570,"hit_rate":16.6667,"demand_bytes":4294901760,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":538,"miss":2546,"hit_rate":17.4449,"demand_bytes":4254793728,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":630,"miss":2454,"hit_rate":20.428,"demand_bytes":4503109632,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":615,"miss":2469,"hit_rate":19.9416,"demand_bytes":4126113792,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":599,"miss":2485,"hit_rate":19.4228,"demand_bytes":4559994880,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":677,"miss":2407,"hit_rate":21.952,"demand_bytes":4022501376,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":692,"miss":2392,"hit_rate":22.4384,"demand_bytes":3997433856,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":505,"miss":2579,"hit_rate":16.3748,"demand_bytes":4309942272,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":551,"miss":2533,"hit_rate":17.8664,"demand_bytes":4233068544,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":448,"miss":2636,"hit_rate":14.5266,"demand_bytes":4405198848,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":433,"miss":2651,"hit_rate":14.0402,"demand_bytes":4430266368,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":576,"miss":2508,"hit_rate":18.677,"demand_bytes":4602200064,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":496,"miss":2588,"hit_rate":16.083,"demand_bytes":4324982784,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":298,"miss":1244,"hit_rate":19.3256,"demand_bytes":2078932992,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":25819,"expert_misses":117587,"hit_rate":18.0041,"ssb_demand_bytes":225806499840,"ssb_prefetch_bytes":0,"ssb_total_bytes":225806499840,"ssd_bytes_per_token":4.39312e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":117167,"demand_wait_us":282251352,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":1199005696,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":77,"miss":3007,"hit_rate":2.49676,"demand_bytes":4877402112,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":117,"miss":2967,"hit_rate":3.79377,"demand_bytes":6416695296,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":278,"miss":2806,"hit_rate":9.01427,"demand_bytes":7171866624,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":294,"miss":2790,"hit_rate":9.53307,"demand_bytes":6033899520,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":314,"miss":2770,"hit_rate":10.1816,"demand_bytes":5990645760,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":425,"miss":2659,"hit_rate":13.7808,"demand_bytes":5750587392,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":616,"miss":2468,"hit_rate":19.9741,"demand_bytes":5681217536,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":551,"miss":2533,"hit_rate":17.8664,"demand_bytes":5478088704,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":420,"miss":2664,"hit_rate":13.6187,"demand_bytes":5761400832,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":541,"miss":2543,"hit_rate":17.5422,"demand_bytes":5499715584,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":594,"miss":2490,"hit_rate":19.2607,"demand_bytes":5385093120,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":568,"miss":2516,"hit_rate":18.4176,"demand_bytes":5441323008,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":693,"miss":2391,"hit_rate":22.4708,"demand_bytes":5170987008,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":672,"miss":2412,"hit_rate":21.7899,"demand_bytes":5216403456,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":747,"miss":2337,"hit_rate":24.2218,"demand_bytes":5054201856,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":677,"miss":2407,"hit_rate":21.952,"demand_bytes":5205590016,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":686,"miss":2398,"hit_rate":22.2438,"demand_bytes":5186125824,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":694,"miss":2390,"hit_rate":22.5032,"demand_bytes":3994091520,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":714,"miss":2370,"hit_rate":23.1518,"demand_bytes":4348968960,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":654,"miss":2430,"hit_rate":21.2062,"demand_bytes":4618321920,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":619,"miss":2465,"hit_rate":20.0713,"demand_bytes":4684840960,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":664,"miss":2420,"hit_rate":21.5305,"demand_bytes":4599316480,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":718,"miss":2366,"hit_rate":23.2815,"demand_bytes":4341628928,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":490,"miss":2594,"hit_rate":15.8885,"demand_bytes":4760010752,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":538,"miss":2546,"hit_rate":17.4449,"demand_bytes":4671930368,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":456,"miss":2628,"hit_rate":14.786,"demand_bytes":5683544064,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":539,"miss":2545,"hit_rate":17.4773,"demand_bytes":4253122560,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":753,"miss":2331,"hit_rate":24.4163,"demand_bytes":3895492608,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":575,"miss":2509,"hit_rate":18.6446,"demand_bytes":4192960512,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":550,"miss":2534,"hit_rate":17.834,"demand_bytes":4234739712,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":706,"miss":2378,"hit_rate":22.8923,"demand_bytes":5142872064,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":678,"miss":2406,"hit_rate":21.9844,"demand_bytes":4415029248,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":629,"miss":2455,"hit_rate":20.3956,"demand_bytes":4504944640,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":514,"miss":2570,"hit_rate":16.6667,"demand_bytes":4294901760,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":538,"miss":2546,"hit_rate":17.4449,"demand_bytes":4254793728,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":630,"miss":2454,"hit_rate":20.428,"demand_bytes":4503109632,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":615,"miss":2469,"hit_rate":19.9416,"demand_bytes":4126113792,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":599,"miss":2485,"hit_rate":19.4228,"demand_bytes":4559994880,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":677,"miss":2407,"hit_rate":21.952,"demand_bytes":4022501376,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":692,"miss":2392,"hit_rate":22.4384,"demand_bytes":3997433856,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":505,"miss":2579,"hit_rate":16.3748,"demand_bytes":4309942272,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":551,"miss":2533,"hit_rate":17.8664,"demand_bytes":4233068544,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":448,"miss":2636,"hit_rate":14.5266,"demand_bytes":4405198848,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":433,"miss":2651,"hit_rate":14.0402,"demand_bytes":4430266368,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":576,"miss":2508,"hit_rate":18.677,"demand_bytes":4602200064,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":496,"miss":2588,"hit_rate":16.083,"demand_bytes":4324982784,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":298,"miss":1244,"hit_rate":19.3256,"demand_bytes":2078932992,"prefetch_bytes":0}]}

results/hot/stats_lru_128.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":0,"expert_misses":558,"hit_rate":0,"ssb_demand_bytes":1070432256,"ssb_prefetch_bytes":0,"ssb_total_bytes":1070432256,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":506,"demand_wait_us":1314554,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":1958277120,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":30670848,"prefetch_bytes":0},{"layer":5,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":10,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":11,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":12,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":13,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":16,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":17,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":18,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":19,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":45,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":46,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":0,"expert_misses":71982,"hit_rate":0,"ssb_demand_bytes":138085761024,"ssb_prefetch_bytes":0,"ssb_total_bytes":138085761024,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":71930,"demand_wait_us":212979253,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":2046648320,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2510880768,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3956539392,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3563421696,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":48,"req":774,"hit":0,"miss":774,"hit_rate":0,"demand_bytes":1293484032,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":0,"expert_misses":143406,"hit_rate":0,"ssb_demand_bytes":275101089792,"ssb_prefetch_bytes":0,"ssb_total_bytes":275101089792,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":143354,"demand_wait_us":440441288,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":2048221184,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7099219968,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":0,"expert_misses":143406,"hit_rate":0,"ssb_demand_bytes":275101089792,"ssb_prefetch_bytes":0,"ssb_total_bytes":275101089792,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":143354,"demand_wait_us":440441288,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":255025152,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7099219968,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}

results/hot/stats_lru_256.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":0,"expert_misses":558,"hit_rate":0,"ssb_demand_bytes":1070432256,"ssb_prefetch_bytes":0,"ssb_total_bytes":1070432256,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":453,"demand_wait_us":1338583,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":2089218048,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":30670848,"prefetch_bytes":0},{"layer":5,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":10,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":11,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":12,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":13,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":16,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":17,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":18,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":19,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":45,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":46,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":0,"expert_misses":71982,"hit_rate":0,"ssb_demand_bytes":138085761024,"ssb_prefetch_bytes":0,"ssb_total_bytes":138085761024,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":71877,"demand_wait_us":219300964,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":2177634304,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2510880768,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3956539392,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3563421696,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":48,"req":774,"hit":0,"miss":774,"hit_rate":0,"demand_bytes":1293484032,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":0,"expert_misses":143406,"hit_rate":0,"ssb_demand_bytes":275101089792,"ssb_prefetch_bytes":0,"ssb_total_bytes":275101089792,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":143301,"demand_wait_us":439271481,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":2179612672,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7099219968,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":0,"expert_misses":143406,"hit_rate":0,"ssb_demand_bytes":275101089792,"ssb_prefetch_bytes":0,"ssb_total_bytes":275101089792,"ssd_bytes_per_token":5.35216e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":143301,"demand_wait_us":439271481,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":393777152,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7099219968,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}

results/hot/stats_lru_3072.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":23,"expert_misses":535,"hit_rate":4.12186,"ssb_demand_bytes":1022918656,"ssb_prefetch_bytes":0,"ssb_total_bytes":1022918656,"ssd_bytes_per_token":5.11459e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":0,"demand_wait_us":1512625,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":1367408640,"rss_bytes":5045862400,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":28114944,"prefetch_bytes":0},{"layer":5,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":10,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":11,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":12,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":13,"req":12,"hit":3,"miss":9,"hit_rate":25,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":16,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":17,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":18,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":19,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":45,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":46,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":27368,"expert_misses":44614,"hit_rate":38.0206,"ssb_demand_bytes":85886377984,"ssb_prefetch_bytes":0,"ssb_total_bytes":85886377984,"ssd_bytes_per_token":3.32893e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":43354,"demand_wait_us":100340284,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":3220439040,"rss_bytes":5134336000,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":284,"miss":1264,"hit_rate":18.3463,"demand_bytes":2050228224,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":298,"miss":1250,"hit_rate":19.2506,"demand_bytes":2703360000,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":382,"miss":1166,"hit_rate":24.677,"demand_bytes":2980184064,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":459,"miss":1089,"hit_rate":29.6512,"demand_bytes":2355167232,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":431,"miss":1117,"hit_rate":27.8424,"demand_bytes":2415722496,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":485,"miss":1063,"hit_rate":31.3307,"demand_bytes":2298937344,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":647,"miss":901,"hit_rate":41.7959,"demand_bytes":2074058752,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":506,"miss":1042,"hit_rate":32.6873,"demand_bytes":2253520896,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":493,"miss":1055,"hit_rate":31.8475,"demand_bytes":2281635840,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":544,"miss":1004,"hit_rate":35.1421,"demand_bytes":2171338752,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":583,"miss":965,"hit_rate":37.6615,"demand_bytes":2086993920,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":591,"miss":957,"hit_rate":38.1783,"demand_bytes":2069692416,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":619,"miss":929,"hit_rate":39.9871,"demand_bytes":2009137152,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":673,"miss":875,"hit_rate":43.4755,"demand_bytes":1892352000,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":691,"miss":857,"hit_rate":44.6382,"demand_bytes":1853423616,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":677,"miss":871,"hit_rate":43.7339,"demand_bytes":1883701248,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":659,"miss":889,"hit_rate":42.5711,"demand_bytes":1922629632,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":695,"miss":853,"hit_rate":44.8966,"demand_bytes":1425506304,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":683,"miss":865,"hit_rate":44.1214,"demand_bytes":1587281920,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":646,"miss":902,"hit_rate":41.7313,"demand_bytes":1714290688,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":593,"miss":955,"hit_rate":38.3075,"demand_bytes":1815019520,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":630,"miss":918,"hit_rate":40.6977,"demand_bytes":1744699392,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":609,"miss":939,"hit_rate":39.3411,"demand_bytes":1723072512,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":538,"miss":1010,"hit_rate":34.7545,"demand_bytes":1853358080,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":555,"miss":993,"hit_rate":35.8527,"demand_bytes":1822162944,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":519,"miss":1029,"hit_rate":33.5271,"demand_bytes":2225405952,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":550,"miss":998,"hit_rate":35.5297,"demand_bytes":1667825664,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":686,"miss":862,"hit_rate":44.3152,"demand_bytes":1440546816,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":612,"miss":936,"hit_rate":39.5349,"demand_bytes":1564213248,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":573,"miss":975,"hit_rate":37.0155,"demand_bytes":1629388800,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":651,"miss":897,"hit_rate":42.0543,"demand_bytes":1939931136,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":659,"miss":889,"hit_rate":42.5711,"demand_bytes":1631322112,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":639,"miss":909,"hit_rate":41.2791,"demand_bytes":1668022272,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":614,"miss":934,"hit_rate":39.6641,"demand_bytes":1560870912,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":630,"miss":918,"hit_rate":40.6977,"demand_bytes":1534132224,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":695,"miss":853,"hit_rate":44.8966,"demand_bytes":1565261824,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":655,"miss":893,"hit_rate":42.3127,"demand_bytes":1492353024,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":676,"miss":872,"hit_rate":43.6693,"demand_bytes":1600126976,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":656,"miss":892,"hit_rate":42.3773,"demand_bytes":1490681856,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":665,"miss":883,"hit_rate":42.9587,"demand_bytes":1475641344,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":606,"miss":942,"hit_rate":39.1473,"demand_bytes":1574240256,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":618,"miss":930,"hit_rate":39.9225,"demand_bytes":1554186240,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":535,"miss":1013,"hit_rate":34.5607,"demand_bytes":1692893184,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":571,"miss":977,"hit_rate":36.8863,"demand_bytes":1632731136,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":670,"miss":878,"hit_rate":43.2817,"demand_bytes":1611137024,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":592,"miss":956,"hit_rate":38.2429,"demand_bytes":1597636608,"prefetch_bytes":0},{"layer":48,"req":774,"hit":325,"miss":449,"hit_rate":41.9897,"demand_bytes":750354432,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":54436,"expert_misses":88970,"hit_rate":37.9594,"ssb_demand_bytes":171037876224,"ssb_prefetch_bytes":0,"ssb_total_bytes":171037876224,"ssd_bytes_per_token":3.32759e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":87710,"demand_wait_us":195232272,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":3220439040,"rss_bytes":5136719872,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":585,"miss":2499,"hit_rate":18.9689,"demand_bytes":4053417984,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":629,"miss":2455,"hit_rate":20.3956,"demand_bytes":5309399040,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":793,"miss":2291,"hit_rate":25.7134,"demand_bytes":5855576064,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":824,"miss":2260,"hit_rate":26.7185,"demand_bytes":4887674880,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":885,"miss":2199,"hit_rate":28.6965,"demand_bytes":4755750912,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":997,"miss":2087,"hit_rate":32.3281,"demand_bytes":4513529856,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":1303,"miss":1781,"hit_rate":42.2503,"demand_bytes":4099776512,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":1138,"miss":1946,"hit_rate":36.9001,"demand_bytes":4208590848,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":985,"miss":2099,"hit_rate":31.939,"demand_bytes":4539482112,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":1118,"miss":1966,"hit_rate":36.2516,"demand_bytes":4251844608,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":1212,"miss":1872,"hit_rate":39.2996,"demand_bytes":4048551936,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":1203,"miss":1881,"hit_rate":39.0078,"demand_bytes":4068016128,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":1231,"miss":1853,"hit_rate":39.9157,"demand_bytes":4007460864,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":1380,"miss":1704,"hit_rate":44.7471,"demand_bytes":3685220352,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":1365,"miss":1719,"hit_rate":44.2607,"demand_bytes":3717660672,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":1357,"miss":1727,"hit_rate":44.0013,"demand_bytes":3734962176,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":1401,"miss":1683,"hit_rate":45.428,"demand_bytes":3639803904,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":1414,"miss":1670,"hit_rate":45.8495,"demand_bytes":2790850560,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":1457,"miss":1627,"hit_rate":47.2438,"demand_bytes":2985558016,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":1287,"miss":1797,"hit_rate":41.7315,"demand_bytes":3415277568,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":1254,"miss":1830,"hit_rate":40.6615,"demand_bytes":3477995520,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":1234,"miss":1850,"hit_rate":40.013,"demand_bytes":3516006400,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":1250,"miss":1834,"hit_rate":40.5318,"demand_bytes":3365404672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":1111,"miss":1973,"hit_rate":36.0246,"demand_bytes":3620470784,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":1136,"miss":1948,"hit_rate":36.8353,"demand_bytes":3574595584,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":1016,"miss":2068,"hit_rate":32.9442,"demand_bytes":4472438784,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":1062,"miss":2022,"hit_rate":34.4358,"demand_bytes":3379101696,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":1287,"miss":1797,"hit_rate":41.7315,"demand_bytes":3003088896,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":1177,"miss":1907,"hit_rate":38.1647,"demand_bytes":3186917376,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":1168,"miss":1916,"hit_rate":37.8729,"demand_bytes":3201957888,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":1311,"miss":1773,"hit_rate":42.5097,"demand_bytes":3834445824,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":1256,"miss":1828,"hit_rate":40.7263,"demand_bytes":3354394624,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":1268,"miss":1816,"hit_rate":41.1154,"demand_bytes":3332374528,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":1170,"miss":1914,"hit_rate":37.9377,"demand_bytes":3198615552,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":1200,"miss":1884,"hit_rate":38.9105,"demand_bytes":3148480512,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":1297,"miss":1787,"hit_rate":42.0558,"demand_bytes":3279159296,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":1273,"miss":1811,"hit_rate":41.2776,"demand_bytes":3026485248,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":1309,"miss":1775,"hit_rate":42.4449,"demand_bytes":3257139200,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":1332,"miss":1752,"hit_rate":43.1907,"demand_bytes":2927886336,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":1297,"miss":1787,"hit_rate":42.0558,"demand_bytes":2986377216,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":1092,"miss":1992,"hit_rate":35.4086,"demand_bytes":3328966656,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":1214,"miss":1870,"hit_rate":39.3645,"demand_bytes":3125084160,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":1042,"miss":2042,"hit_rate":33.7873,"demand_bytes":3412525056,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":1072,"miss":2012,"hit_rate":34.7601,"demand_bytes":3362390016,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":1256,"miss":1828,"hit_rate":40.7263,"demand_bytes":3354394624,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":1166,"miss":1918,"hit_rate":37.808,"demand_bytes":3205300224,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":622,"miss":920,"hit_rate":40.3372,"demand_bytes":1537474560,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":54436,"expert_misses":88970,"hit_rate":37.9594,"ssb_demand_bytes":171037876224,"ssb_prefetch_bytes":0,"ssb_total_bytes":171037876224,"ssd_bytes_per_token":3.32759e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":87710,"demand_wait_us":195232272,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":3220439040,"rss_bytes":3343212544,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":585,"miss":2499,"hit_rate":18.9689,"demand_bytes":4053417984,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":629,"miss":2455,"hit_rate":20.3956,"demand_bytes":5309399040,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":793,"miss":2291,"hit_rate":25.7134,"demand_bytes":5855576064,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":824,"miss":2260,"hit_rate":26.7185,"demand_bytes":4887674880,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":885,"miss":2199,"hit_rate":28.6965,"demand_bytes":4755750912,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":997,"miss":2087,"hit_rate":32.3281,"demand_bytes":4513529856,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":1303,"miss":1781,"hit_rate":42.2503,"demand_bytes":4099776512,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":1138,"miss":1946,"hit_rate":36.9001,"demand_bytes":4208590848,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":985,"miss":2099,"hit_rate":31.939,"demand_bytes":4539482112,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":1118,"miss":1966,"hit_rate":36.2516,"demand_bytes":4251844608,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":1212,"miss":1872,"hit_rate":39.2996,"demand_bytes":4048551936,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":1203,"miss":1881,"hit_rate":39.0078,"demand_bytes":4068016128,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":1231,"miss":1853,"hit_rate":39.9157,"demand_bytes":4007460864,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":1380,"miss":1704,"hit_rate":44.7471,"demand_bytes":3685220352,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":1365,"miss":1719,"hit_rate":44.2607,"demand_bytes":3717660672,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":1357,"miss":1727,"hit_rate":44.0013,"demand_bytes":3734962176,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":1401,"miss":1683,"hit_rate":45.428,"demand_bytes":3639803904,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":1414,"miss":1670,"hit_rate":45.8495,"demand_bytes":2790850560,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":1457,"miss":1627,"hit_rate":47.2438,"demand_bytes":2985558016,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":1287,"miss":1797,"hit_rate":41.7315,"demand_bytes":3415277568,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":1254,"miss":1830,"hit_rate":40.6615,"demand_bytes":3477995520,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":1234,"miss":1850,"hit_rate":40.013,"demand_bytes":3516006400,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":1250,"miss":1834,"hit_rate":40.5318,"demand_bytes":3365404672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":1111,"miss":1973,"hit_rate":36.0246,"demand_bytes":3620470784,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":1136,"miss":1948,"hit_rate":36.8353,"demand_bytes":3574595584,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":1016,"miss":2068,"hit_rate":32.9442,"demand_bytes":4472438784,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":1062,"miss":2022,"hit_rate":34.4358,"demand_bytes":3379101696,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":1287,"miss":1797,"hit_rate":41.7315,"demand_bytes":3003088896,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":1177,"miss":1907,"hit_rate":38.1647,"demand_bytes":3186917376,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":1168,"miss":1916,"hit_rate":37.8729,"demand_bytes":3201957888,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":1311,"miss":1773,"hit_rate":42.5097,"demand_bytes":3834445824,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":1256,"miss":1828,"hit_rate":40.7263,"demand_bytes":3354394624,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":1268,"miss":1816,"hit_rate":41.1154,"demand_bytes":3332374528,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":1170,"miss":1914,"hit_rate":37.9377,"demand_bytes":3198615552,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":1200,"miss":1884,"hit_rate":38.9105,"demand_bytes":3148480512,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":1297,"miss":1787,"hit_rate":42.0558,"demand_bytes":3279159296,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":1273,"miss":1811,"hit_rate":41.2776,"demand_bytes":3026485248,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":1309,"miss":1775,"hit_rate":42.4449,"demand_bytes":3257139200,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":1332,"miss":1752,"hit_rate":43.1907,"demand_bytes":2927886336,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":1297,"miss":1787,"hit_rate":42.0558,"demand_bytes":2986377216,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":1092,"miss":1992,"hit_rate":35.4086,"demand_bytes":3328966656,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":1214,"miss":1870,"hit_rate":39.3645,"demand_bytes":3125084160,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":1042,"miss":2042,"hit_rate":33.7873,"demand_bytes":3412525056,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":1072,"miss":2012,"hit_rate":34.7601,"demand_bytes":3362390016,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":1256,"miss":1828,"hit_rate":40.7263,"demand_bytes":3354394624,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":1166,"miss":1918,"hit_rate":37.808,"demand_bytes":3205300224,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":622,"miss":920,"hit_rate":40.3372,"demand_bytes":1537474560,"prefetch_bytes":0}]}

results/hot/stats_pin_1024.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":23,"expert_misses":535,"hit_rate":4.12186,"ssb_demand_bytes":1022918656,"ssb_prefetch_bytes":0,"ssb_total_bytes":1022918656,"ssd_bytes_per_token":5.11459e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":115,"demand_wait_us":1300981,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":2894479360,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":28114944,"prefetch_bytes":0},{"layer":5,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":10,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":11,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":12,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":13,"req":12,"hit":3,"miss":9,"hit_rate":25,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":16,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":17,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":18,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":19,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":45,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":46,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":15642,"expert_misses":56340,"hit_rate":21.7304,"ssb_demand_bytes":108319981568,"ssb_prefetch_bytes":0,"ssb_total_bytes":108319981568,"ssd_bytes_per_token":4.19845e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":55920,"demand_wait_us":134154306,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":2982871040,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":37,"miss":1511,"hit_rate":2.39018,"demand_bytes":2450866176,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":52,"miss":1496,"hit_rate":3.35917,"demand_bytes":3235381248,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":129,"miss":1419,"hit_rate":8.33333,"demand_bytes":3626827776,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":261,"miss":1287,"hit_rate":16.8605,"demand_bytes":2783379456,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":171,"miss":1377,"hit_rate":11.0465,"demand_bytes":2978021376,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":260,"miss":1288,"hit_rate":16.7959,"demand_bytes":2785542144,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":388,"miss":1160,"hit_rate":25.0646,"demand_bytes":2670264320,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":288,"miss":1260,"hit_rate":18.6047,"demand_bytes":2724986880,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":218,"miss":1330,"hit_rate":14.0827,"demand_bytes":2876375040,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":322,"miss":1226,"hit_rate":20.801,"demand_bytes":2651455488,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":322,"miss":1226,"hit_rate":20.801,"demand_bytes":2651455488,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":325,"miss":1223,"hit_rate":20.9948,"demand_bytes":2644967424,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":422,"miss":1126,"hit_rate":27.261,"demand_bytes":2435186688,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":379,"miss":1169,"hit_rate":24.4832,"demand_bytes":2528182272,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":410,"miss":1138,"hit_rate":26.4858,"demand_bytes":2461138944,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":419,"miss":1129,"hit_rate":27.0672,"demand_bytes":2441674752,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":437,"miss":1111,"hit_rate":28.23,"demand_bytes":2402746368,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":474,"miss":1074,"hit_rate":30.6202,"demand_bytes":1794834432,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":473,"miss":1075,"hit_rate":30.5556,"demand_bytes":1972633600,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":411,"miss":1137,"hit_rate":26.5504,"demand_bytes":2160918528,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":295,"miss":1253,"hit_rate":19.0568,"demand_bytes":2381381632,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":452,"miss":1096,"hit_rate":29.199,"demand_bytes":2082996224,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":364,"miss":1184,"hit_rate":23.5142,"demand_bytes":2172649472,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":244,"miss":1304,"hit_rate":15.7623,"demand_bytes":2392850432,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":311,"miss":1237,"hit_rate":20.0904,"demand_bytes":2269904896,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":277,"miss":1271,"hit_rate":17.8941,"demand_bytes":2748776448,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":349,"miss":1199,"hit_rate":22.5452,"demand_bytes":2003730432,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":506,"miss":1042,"hit_rate":32.6873,"demand_bytes":1741357056,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":396,"miss":1152,"hit_rate":25.5814,"demand_bytes":1925185536,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":301,"miss":1247,"hit_rate":19.4444,"demand_bytes":2083946496,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":389,"miss":1159,"hit_rate":25.1292,"demand_bytes":2506555392,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":477,"miss":1071,"hit_rate":30.814,"demand_bytes":1965293568,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":427,"miss":1121,"hit_rate":27.584,"demand_bytes":2057043968,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":373,"miss":1175,"hit_rate":24.0956,"demand_bytes":1963622400,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":376,"miss":1172,"hit_rate":24.2894,"demand_bytes":1958608896,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":420,"miss":1128,"hit_rate":27.1318,"demand_bytes":2069889024,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":337,"miss":1211,"hit_rate":21.77,"demand_bytes":2023784448,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":395,"miss":1153,"hit_rate":25.5168,"demand_bytes":2115764224,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":401,"miss":1147,"hit_rate":25.9044,"demand_bytes":1916829696,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":399,"miss":1149,"hit_rate":25.7752,"demand_bytes":1920172032,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":326,"miss":1222,"hit_rate":21.0594,"demand_bytes":2042167296,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":297,"miss":1251,"hit_rate":19.186,"demand_bytes":2090631168,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":231,"miss":1317,"hit_rate":14.9225,"demand_bytes":2200928256,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":285,"miss":1263,"hit_rate":18.4109,"demand_bytes":2110685184,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":360,"miss":1188,"hit_rate":23.2558,"demand_bytes":2179989504,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":272,"miss":1276,"hit_rate":17.5711,"demand_bytes":2132410368,"prefetch_bytes":0},{"layer":48,"req":774,"hit":184,"miss":590,"hit_rate":23.7726,"demand_bytes":985989120,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":30071,"expert_misses":113335,"hit_rate":20.9691,"ssb_demand_bytes":217696280576,"ssb_prefetch_bytes":0,"ssb_total_bytes":217696280576,"ssd_bytes_per_token":4.23534e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":112915,"demand_wait_us":275716152,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":2984751104,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":77,"miss":3007,"hit_rate":2.49676,"demand_bytes":4877402112,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":117,"miss":2967,"hit_rate":3.79377,"demand_bytes":6416695296,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":278,"miss":2806,"hit_rate":9.01427,"demand_bytes":7171866624,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":399,"miss":2685,"hit_rate":12.9377,"demand_bytes":5806817280,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":356,"miss":2728,"hit_rate":11.5435,"demand_bytes":5899812864,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":535,"miss":2549,"hit_rate":17.3476,"demand_bytes":5512691712,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":777,"miss":2307,"hit_rate":25.1946,"demand_bytes":5310603264,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":639,"miss":2445,"hit_rate":20.7198,"demand_bytes":5287772160,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":445,"miss":2639,"hit_rate":14.4293,"demand_bytes":5707333632,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":627,"miss":2457,"hit_rate":20.3307,"demand_bytes":5313724416,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":649,"miss":2435,"hit_rate":21.0441,"demand_bytes":5266145280,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":651,"miss":2433,"hit_rate":21.1089,"demand_bytes":5261819904,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":796,"miss":2288,"hit_rate":25.8106,"demand_bytes":4948230144,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":777,"miss":2307,"hit_rate":25.1946,"demand_bytes":4989321216,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":789,"miss":2295,"hit_rate":25.5837,"demand_bytes":4963368960,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":811,"miss":2273,"hit_rate":26.297,"demand_bytes":4915789824,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":908,"miss":2176,"hit_rate":29.4423,"demand_bytes":4706009088,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":944,"miss":2140,"hit_rate":30.6096,"demand_bytes":3576299520,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":1064,"miss":2020,"hit_rate":34.5006,"demand_bytes":3706716160,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":814,"miss":2270,"hit_rate":26.3943,"demand_bytes":4314234880,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":632,"miss":2452,"hit_rate":20.4929,"demand_bytes":4660133888,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":850,"miss":2234,"hit_rate":27.5616,"demand_bytes":4245815296,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":731,"miss":2353,"hit_rate":23.703,"demand_bytes":4317773824,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":507,"miss":2577,"hit_rate":16.4397,"demand_bytes":4728815616,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":622,"miss":2462,"hit_rate":20.1686,"demand_bytes":4517789696,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":512,"miss":2572,"hit_rate":16.6018,"demand_bytes":5562433536,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":630,"miss":2454,"hit_rate":20.428,"demand_bytes":4101046272,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":901,"miss":2183,"hit_rate":29.2153,"demand_bytes":3648159744,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":690,"miss":2394,"hit_rate":22.3735,"demand_bytes":4000776192,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":602,"miss":2482,"hit_rate":19.5201,"demand_bytes":4147838976,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":755,"miss":2329,"hit_rate":24.4812,"demand_bytes":5036900352,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":850,"miss":2234,"hit_rate":27.5616,"demand_bytes":4099407872,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":802,"miss":2282,"hit_rate":26.0052,"demand_bytes":4187488256,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":676,"miss":2408,"hit_rate":21.9196,"demand_bytes":4024172544,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":638,"miss":2446,"hit_rate":20.6874,"demand_bytes":4087676928,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":728,"miss":2356,"hit_rate":23.6057,"demand_bytes":4323278848,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":615,"miss":2469,"hit_rate":19.9416,"demand_bytes":4126113792,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":709,"miss":2375,"hit_rate":22.9896,"demand_bytes":4358144000,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":820,"miss":2264,"hit_rate":26.5888,"demand_bytes":3783524352,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":749,"miss":2335,"hit_rate":24.2866,"demand_bytes":3902177280,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":579,"miss":2505,"hit_rate":18.7743,"demand_bytes":4186275840,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":570,"miss":2514,"hit_rate":18.4825,"demand_bytes":4201316352,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":448,"miss":2636,"hit_rate":14.5266,"demand_bytes":4405198848,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":504,"miss":2580,"hit_rate":16.3424,"demand_bytes":4311613440,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":652,"miss":2432,"hit_rate":21.1414,"demand_bytes":4462739456,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":515,"miss":2569,"hit_rate":16.6991,"demand_bytes":4293230592,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":331,"miss":1211,"hit_rate":21.4656,"demand_bytes":2023784448,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":30071,"expert_misses":113335,"hit_rate":20.9691,"ssb_demand_bytes":217696280576,"ssb_prefetch_bytes":0,"ssb_total_bytes":217696280576,"ssd_bytes_per_token":4.23534e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":112915,"demand_wait_us":275716152,"prefetch_wait_us":0,"arena_bytes":1073479680,"arena_used_bytes":1073479680,"rss_bytes":1199046656,"n_slots":420,"n_small_slots":0,"n_big_slots":420,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":77,"miss":3007,"hit_rate":2.49676,"demand_bytes":4877402112,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":117,"miss":2967,"hit_rate":3.79377,"demand_bytes":6416695296,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":278,"miss":2806,"hit_rate":9.01427,"demand_bytes":7171866624,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":399,"miss":2685,"hit_rate":12.9377,"demand_bytes":5806817280,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":356,"miss":2728,"hit_rate":11.5435,"demand_bytes":5899812864,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":535,"miss":2549,"hit_rate":17.3476,"demand_bytes":5512691712,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":777,"miss":2307,"hit_rate":25.1946,"demand_bytes":5310603264,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":639,"miss":2445,"hit_rate":20.7198,"demand_bytes":5287772160,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":445,"miss":2639,"hit_rate":14.4293,"demand_bytes":5707333632,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":627,"miss":2457,"hit_rate":20.3307,"demand_bytes":5313724416,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":649,"miss":2435,"hit_rate":21.0441,"demand_bytes":5266145280,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":651,"miss":2433,"hit_rate":21.1089,"demand_bytes":5261819904,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":796,"miss":2288,"hit_rate":25.8106,"demand_bytes":4948230144,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":777,"miss":2307,"hit_rate":25.1946,"demand_bytes":4989321216,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":789,"miss":2295,"hit_rate":25.5837,"demand_bytes":4963368960,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":811,"miss":2273,"hit_rate":26.297,"demand_bytes":4915789824,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":908,"miss":2176,"hit_rate":29.4423,"demand_bytes":4706009088,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":944,"miss":2140,"hit_rate":30.6096,"demand_bytes":3576299520,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":1064,"miss":2020,"hit_rate":34.5006,"demand_bytes":3706716160,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":814,"miss":2270,"hit_rate":26.3943,"demand_bytes":4314234880,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":632,"miss":2452,"hit_rate":20.4929,"demand_bytes":4660133888,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":850,"miss":2234,"hit_rate":27.5616,"demand_bytes":4245815296,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":731,"miss":2353,"hit_rate":23.703,"demand_bytes":4317773824,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":507,"miss":2577,"hit_rate":16.4397,"demand_bytes":4728815616,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":622,"miss":2462,"hit_rate":20.1686,"demand_bytes":4517789696,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":512,"miss":2572,"hit_rate":16.6018,"demand_bytes":5562433536,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":630,"miss":2454,"hit_rate":20.428,"demand_bytes":4101046272,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":901,"miss":2183,"hit_rate":29.2153,"demand_bytes":3648159744,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":690,"miss":2394,"hit_rate":22.3735,"demand_bytes":4000776192,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":602,"miss":2482,"hit_rate":19.5201,"demand_bytes":4147838976,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":755,"miss":2329,"hit_rate":24.4812,"demand_bytes":5036900352,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":850,"miss":2234,"hit_rate":27.5616,"demand_bytes":4099407872,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":802,"miss":2282,"hit_rate":26.0052,"demand_bytes":4187488256,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":676,"miss":2408,"hit_rate":21.9196,"demand_bytes":4024172544,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":638,"miss":2446,"hit_rate":20.6874,"demand_bytes":4087676928,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":728,"miss":2356,"hit_rate":23.6057,"demand_bytes":4323278848,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":615,"miss":2469,"hit_rate":19.9416,"demand_bytes":4126113792,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":709,"miss":2375,"hit_rate":22.9896,"demand_bytes":4358144000,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":820,"miss":2264,"hit_rate":26.5888,"demand_bytes":3783524352,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":749,"miss":2335,"hit_rate":24.2866,"demand_bytes":3902177280,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":579,"miss":2505,"hit_rate":18.7743,"demand_bytes":4186275840,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":570,"miss":2514,"hit_rate":18.4825,"demand_bytes":4201316352,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":448,"miss":2636,"hit_rate":14.5266,"demand_bytes":4405198848,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":504,"miss":2580,"hit_rate":16.3424,"demand_bytes":4311613440,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":652,"miss":2432,"hit_rate":21.1414,"demand_bytes":4462739456,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":515,"miss":2569,"hit_rate":16.6991,"demand_bytes":4293230592,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":331,"miss":1211,"hit_rate":21.4656,"demand_bytes":2023784448,"prefetch_bytes":0}]}

results/hot/stats_pin_128.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":2,"expert_misses":556,"hit_rate":0.358423,"ssb_demand_bytes":1066106880,"ssb_prefetch_bytes":0,"ssb_total_bytes":1066106880,"ssd_bytes_per_token":5.33053e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":504,"demand_wait_us":1267960,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":1958158336,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":30670848,"prefetch_bytes":0},{"layer":5,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":10,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":11,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":12,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":13,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":16,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":17,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":18,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":19,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":45,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":46,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":1300,"expert_misses":70682,"hit_rate":1.80601,"ssb_demand_bytes":135582228480,"ssb_prefetch_bytes":0,"ssb_total_bytes":135582228480,"ssd_bytes_per_token":5.25513e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":70630,"demand_wait_us":205258668,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":2046439424,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2510880768,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3956539392,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":157,"miss":1391,"hit_rate":10.1421,"demand_bytes":3008299008,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":91,"miss":1457,"hit_rate":5.87855,"demand_bytes":3353944064,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":67,"miss":1481,"hit_rate":4.32817,"demand_bytes":3202940928,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":71,"miss":1477,"hit_rate":4.58656,"demand_bytes":3194290176,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":12,"miss":1536,"hit_rate":0.775194,"demand_bytes":2566914048,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":69,"miss":1479,"hit_rate":4.45736,"demand_bytes":3198615552,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":10,"miss":1538,"hit_rate":0.645995,"demand_bytes":2570256384,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":193,"miss":1355,"hit_rate":12.4677,"demand_bytes":2486435840,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":149,"miss":1399,"hit_rate":9.62532,"demand_bytes":2567176192,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":129,"miss":1419,"hit_rate":8.33333,"demand_bytes":2603876352,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":108,"miss":1440,"hit_rate":6.97674,"demand_bytes":2406481920,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":137,"miss":1411,"hit_rate":8.85013,"demand_bytes":2358018048,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":107,"miss":1441,"hit_rate":6.91214,"demand_bytes":2644246528,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":48,"req":774,"hit":0,"miss":774,"hit_rate":0,"demand_bytes":1293484032,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":1933,"expert_misses":141473,"hit_rate":1.34792,"ssb_demand_bytes":271407079424,"ssb_prefetch_bytes":0,"ssb_total_bytes":271407079424,"ssd_bytes_per_token":5.28029e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":141421,"demand_wait_us":413875200,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":2048339968,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":6237192192,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":121,"miss":2963,"hit_rate":3.92348,"demand_bytes":6820683776,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":96,"miss":2988,"hit_rate":3.11284,"demand_bytes":6462111744,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":106,"miss":2978,"hit_rate":3.43709,"demand_bytes":6440484864,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":19,"miss":3065,"hit_rate":0.616083,"demand_bytes":5122129920,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":105,"miss":2979,"hit_rate":3.40467,"demand_bytes":6442647552,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":15,"miss":3069,"hit_rate":0.486381,"demand_bytes":5128814592,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":243,"miss":2841,"hit_rate":7.87938,"demand_bytes":5213257728,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":230,"miss":2854,"hit_rate":7.45785,"demand_bytes":5237112832,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":170,"miss":2914,"hit_rate":5.51232,"demand_bytes":5347213312,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":190,"miss":2894,"hit_rate":6.16083,"demand_bytes":4836360192,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":238,"miss":2846,"hit_rate":7.71725,"demand_bytes":4756144128,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":5292163072,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":1933,"expert_misses":141473,"hit_rate":1.34792,"ssb_demand_bytes":271407079424,"ssb_prefetch_bytes":0,"ssb_total_bytes":271407079424,"ssd_bytes_per_token":5.28029e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":141421,"demand_wait_us":413875200,"prefetch_wait_us":0,"arena_bytes":132907008,"arena_used_bytes":132907008,"rss_bytes":254959616,"n_slots":52,"n_small_slots":0,"n_big_slots":52,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":6237192192,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":121,"miss":2963,"hit_rate":3.92348,"demand_bytes":6820683776,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":96,"miss":2988,"hit_rate":3.11284,"demand_bytes":6462111744,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":106,"miss":2978,"hit_rate":3.43709,"demand_bytes":6440484864,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":19,"miss":3065,"hit_rate":0.616083,"demand_bytes":5122129920,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":105,"miss":2979,"hit_rate":3.40467,"demand_bytes":6442647552,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":15,"miss":3069,"hit_rate":0.486381,"demand_bytes":5128814592,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":243,"miss":2841,"hit_rate":7.87938,"demand_bytes":5213257728,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":230,"miss":2854,"hit_rate":7.45785,"demand_bytes":5237112832,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":170,"miss":2914,"hit_rate":5.51232,"demand_bytes":5347213312,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":190,"miss":2894,"hit_rate":6.16083,"demand_bytes":4836360192,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":238,"miss":2846,"hit_rate":7.71725,"demand_bytes":4756144128,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":5292163072,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}

results/hot/stats_pin_256.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":2,"expert_misses":556,"hit_rate":0.358423,"ssb_demand_bytes":1066106880,"ssb_prefetch_bytes":0,"ssb_total_bytes":1066106880,"ssd_bytes_per_token":5.33053e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":451,"demand_wait_us":1737924,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":2093592576,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":30670848,"prefetch_bytes":0},{"layer":5,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":10,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":11,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":12,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":13,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":16,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":17,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":18,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":19,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":45,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":46,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":1937,"expert_misses":70045,"hit_rate":2.69095,"ssb_demand_bytes":134368337920,"ssb_prefetch_bytes":0,"ssb_total_bytes":134368337920,"ssd_bytes_per_token":5.20808e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":69940,"demand_wait_us":204577295,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":2181767168,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2510880768,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3956539392,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":157,"miss":1391,"hit_rate":10.1421,"demand_bytes":3008299008,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":50,"miss":1498,"hit_rate":3.22997,"demand_bytes":3239706624,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":91,"miss":1457,"hit_rate":5.87855,"demand_bytes":3353944064,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":91,"miss":1457,"hit_rate":5.87855,"demand_bytes":3151036416,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":70,"miss":1478,"hit_rate":4.52196,"demand_bytes":3196452864,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":22,"miss":1526,"hit_rate":1.42119,"demand_bytes":3300261888,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":71,"miss":1477,"hit_rate":4.58656,"demand_bytes":3194290176,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":83,"miss":1465,"hit_rate":5.36176,"demand_bytes":2448261120,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":111,"miss":1437,"hit_rate":7.17054,"demand_bytes":2636906496,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":174,"miss":1374,"hit_rate":11.2403,"demand_bytes":2611347456,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2942042112,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2840592384,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":69,"miss":1479,"hit_rate":4.45736,"demand_bytes":3198615552,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":56,"miss":1492,"hit_rate":3.61757,"demand_bytes":2493382656,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":10,"miss":1538,"hit_rate":0.645995,"demand_bytes":2570256384,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":3347841024,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":193,"miss":1355,"hit_rate":12.4677,"demand_bytes":2486435840,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":167,"miss":1381,"hit_rate":10.7881,"demand_bytes":2534146048,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":41,"miss":1507,"hit_rate":2.64858,"demand_bytes":2765357056,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":129,"miss":1419,"hit_rate":8.33333,"demand_bytes":2603876352,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":108,"miss":1440,"hit_rate":6.97674,"demand_bytes":2406481920,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":137,"miss":1411,"hit_rate":8.85013,"demand_bytes":2358018048,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":107,"miss":1441,"hit_rate":6.91214,"demand_bytes":2644246528,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":0,"miss":1548,"hit_rate":0,"demand_bytes":2586968064,"prefetch_bytes":0},{"layer":48,"req":774,"hit":0,"miss":774,"hit_rate":0,"demand_bytes":1293484032,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":3018,"expert_misses":140388,"hit_rate":2.10451,"ssb_demand_bytes":269321986048,"ssb_prefetch_bytes":0,"ssb_total_bytes":269321986048,"ssd_bytes_per_token":5.23973e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":140283,"demand_wait_us":418212435,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":2183712768,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":6237192192,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":112,"miss":2972,"hit_rate":3.63165,"demand_bytes":6427508736,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":121,"miss":2963,"hit_rate":3.92348,"demand_bytes":6820683776,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":125,"miss":2959,"hit_rate":4.05318,"demand_bytes":6399393792,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":124,"miss":2960,"hit_rate":4.02075,"demand_bytes":6401556480,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":47,"miss":3037,"hit_rate":1.52399,"demand_bytes":6568083456,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":106,"miss":2978,"hit_rate":3.43709,"demand_bytes":6440484864,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":126,"miss":2958,"hit_rate":4.0856,"demand_bytes":4943314944,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":215,"miss":2869,"hit_rate":6.97147,"demand_bytes":5264637952,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":306,"miss":2778,"hit_rate":9.92218,"demand_bytes":5279711232,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":105,"miss":2979,"hit_rate":3.40467,"demand_bytes":6442647552,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":65,"miss":3019,"hit_rate":2.10765,"demand_bytes":5045256192,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":15,"miss":3069,"hit_rate":0.486381,"demand_bytes":5128814592,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":243,"miss":2841,"hit_rate":7.87938,"demand_bytes":5213257728,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":251,"miss":2833,"hit_rate":8.13878,"demand_bytes":5198577664,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":59,"miss":3025,"hit_rate":1.9131,"demand_bytes":5550899200,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":170,"miss":2914,"hit_rate":5.51232,"demand_bytes":5347213312,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":190,"miss":2894,"hit_rate":6.16083,"demand_bytes":4836360192,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":238,"miss":2846,"hit_rate":7.71725,"demand_bytes":4756144128,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":5292163072,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":3018,"expert_misses":140388,"hit_rate":2.10451,"ssb_demand_bytes":269321986048,"ssb_prefetch_bytes":0,"ssb_total_bytes":269321986048,"ssd_bytes_per_token":5.23973e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":140283,"demand_wait_us":418212435,"prefetch_wait_us":0,"arena_bytes":268369920,"arena_used_bytes":268369920,"rss_bytes":390471680,"n_slots":105,"n_small_slots":0,"n_big_slots":105,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5002297344,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":7882407936,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":6237192192,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":112,"miss":2972,"hit_rate":3.63165,"demand_bytes":6427508736,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":121,"miss":2963,"hit_rate":3.92348,"demand_bytes":6820683776,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":125,"miss":2959,"hit_rate":4.05318,"demand_bytes":6399393792,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":124,"miss":2960,"hit_rate":4.02075,"demand_bytes":6401556480,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":47,"miss":3037,"hit_rate":1.52399,"demand_bytes":6568083456,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":106,"miss":2978,"hit_rate":3.43709,"demand_bytes":6440484864,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":126,"miss":2958,"hit_rate":4.0856,"demand_bytes":4943314944,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":215,"miss":2869,"hit_rate":6.97147,"demand_bytes":5264637952,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":306,"miss":2778,"hit_rate":9.92218,"demand_bytes":5279711232,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5861277696,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5659164672,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":105,"miss":2979,"hit_rate":3.40467,"demand_bytes":6442647552,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":65,"miss":3019,"hit_rate":2.10765,"demand_bytes":5045256192,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":15,"miss":3069,"hit_rate":0.486381,"demand_bytes":5128814592,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":6669729792,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":243,"miss":2841,"hit_rate":7.87938,"demand_bytes":5213257728,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":251,"miss":2833,"hit_rate":8.13878,"demand_bytes":5198577664,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":59,"miss":3025,"hit_rate":1.9131,"demand_bytes":5550899200,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":170,"miss":2914,"hit_rate":5.51232,"demand_bytes":5347213312,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":190,"miss":2894,"hit_rate":6.16083,"demand_bytes":4836360192,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":238,"miss":2846,"hit_rate":7.71725,"demand_bytes":4756144128,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":200,"miss":2884,"hit_rate":6.48508,"demand_bytes":5292163072,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":0,"miss":3084,"hit_rate":0,"demand_bytes":5153882112,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":0,"miss":1542,"hit_rate":0,"demand_bytes":2576941056,"prefetch_bytes":0}]}

results/hot/stats_pin_3072.jsonl

@@ -0,0 +1,4 @@
+{"phase":"decode","tokens":2,"expert_requests":558,"expert_hits":23,"expert_misses":535,"hit_rate":4.12186,"ssb_demand_bytes":1022918656,"ssb_prefetch_bytes":0,"ssb_total_bytes":1022918656,"ssd_bytes_per_token":5.11459e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":0,"demand_wait_us":1328943,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":1367408640,"rss_bytes":5046132736,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":3,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":4,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":28114944,"prefetch_bytes":0},{"layer":5,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":6,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":7,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":8,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":27623424,"prefetch_bytes":0},{"layer":9,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":10,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":11,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":12,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":13,"req":12,"hit":3,"miss":9,"hit_rate":25,"demand_bytes":19464192,"prefetch_bytes":0},{"layer":14,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":15,"req":12,"hit":2,"miss":10,"hit_rate":16.6667,"demand_bytes":21626880,"prefetch_bytes":0},{"layer":16,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":17,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":18,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":23789568,"prefetch_bytes":0},{"layer":19,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":20,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":21,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":22,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":23,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22806528,"prefetch_bytes":0},{"layer":24,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":25,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":26,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":27,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":28,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":29,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":30,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":31,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":32,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":25952256,"prefetch_bytes":0},{"layer":33,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":34,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":35,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":36,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":37,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":38,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":39,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":22020096,"prefetch_bytes":0},{"layer":40,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":41,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":42,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":43,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":44,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":45,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":18382848,"prefetch_bytes":0},{"layer":46,"req":12,"hit":1,"miss":11,"hit_rate":8.33333,"demand_bytes":20185088,"prefetch_bytes":0},{"layer":47,"req":12,"hit":0,"miss":12,"hit_rate":0,"demand_bytes":20054016,"prefetch_bytes":0},{"layer":48,"req":6,"hit":0,"miss":6,"hit_rate":0,"demand_bytes":10027008,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":258,"expert_requests":71982,"expert_hits":28066,"expert_misses":43916,"hit_rate":38.9903,"ssb_demand_bytes":84604461056,"ssb_prefetch_bytes":0,"ssb_total_bytes":84604461056,"ssd_bytes_per_token":3.27924e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":42656,"demand_wait_us":98450779,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":3220439040,"rss_bytes":5134311424,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":1548,"hit":262,"miss":1286,"hit_rate":16.9251,"demand_bytes":2085912576,"prefetch_bytes":0},{"layer":3,"req":1548,"hit":253,"miss":1295,"hit_rate":16.3437,"demand_bytes":2800680960,"prefetch_bytes":0},{"layer":4,"req":1548,"hit":338,"miss":1210,"hit_rate":21.8346,"demand_bytes":3092643840,"prefetch_bytes":0},{"layer":5,"req":1548,"hit":421,"miss":1127,"hit_rate":27.1964,"demand_bytes":2437349376,"prefetch_bytes":0},{"layer":6,"req":1548,"hit":417,"miss":1131,"hit_rate":26.938,"demand_bytes":2446000128,"prefetch_bytes":0},{"layer":7,"req":1548,"hit":503,"miss":1045,"hit_rate":32.4935,"demand_bytes":2260008960,"prefetch_bytes":0},{"layer":8,"req":1548,"hit":687,"miss":861,"hit_rate":44.3798,"demand_bytes":1981980672,"prefetch_bytes":0},{"layer":9,"req":1548,"hit":481,"miss":1067,"hit_rate":31.0724,"demand_bytes":2307588096,"prefetch_bytes":0},{"layer":10,"req":1548,"hit":493,"miss":1055,"hit_rate":31.8475,"demand_bytes":2281635840,"prefetch_bytes":0},{"layer":11,"req":1548,"hit":556,"miss":992,"hit_rate":35.9173,"demand_bytes":2145386496,"prefetch_bytes":0},{"layer":12,"req":1548,"hit":623,"miss":925,"hit_rate":40.2455,"demand_bytes":2000486400,"prefetch_bytes":0},{"layer":13,"req":1548,"hit":585,"miss":963,"hit_rate":37.7907,"demand_bytes":2082668544,"prefetch_bytes":0},{"layer":14,"req":1548,"hit":646,"miss":902,"hit_rate":41.7313,"demand_bytes":1950744576,"prefetch_bytes":0},{"layer":15,"req":1548,"hit":711,"miss":837,"hit_rate":45.9302,"demand_bytes":1810169856,"prefetch_bytes":0},{"layer":16,"req":1548,"hit":722,"miss":826,"hit_rate":46.6408,"demand_bytes":1786380288,"prefetch_bytes":0},{"layer":17,"req":1548,"hit":690,"miss":858,"hit_rate":44.5736,"demand_bytes":1855586304,"prefetch_bytes":0},{"layer":18,"req":1548,"hit":699,"miss":849,"hit_rate":45.155,"demand_bytes":1836122112,"prefetch_bytes":0},{"layer":19,"req":1548,"hit":751,"miss":797,"hit_rate":48.5142,"demand_bytes":1331920896,"prefetch_bytes":0},{"layer":20,"req":1548,"hit":740,"miss":808,"hit_rate":47.8036,"demand_bytes":1482686464,"prefetch_bytes":0},{"layer":21,"req":1548,"hit":680,"miss":868,"hit_rate":43.9276,"demand_bytes":1649672192,"prefetch_bytes":0},{"layer":22,"req":1548,"hit":634,"miss":914,"hit_rate":40.9561,"demand_bytes":1737097216,"prefetch_bytes":0},{"layer":23,"req":1548,"hit":694,"miss":854,"hit_rate":44.832,"demand_bytes":1623064576,"prefetch_bytes":0},{"layer":24,"req":1548,"hit":640,"miss":908,"hit_rate":41.3437,"demand_bytes":1666187264,"prefetch_bytes":0},{"layer":25,"req":1548,"hit":532,"miss":1016,"hit_rate":34.3669,"demand_bytes":1864368128,"prefetch_bytes":0},{"layer":26,"req":1548,"hit":560,"miss":988,"hit_rate":36.1757,"demand_bytes":1812987904,"prefetch_bytes":0},{"layer":27,"req":1548,"hit":574,"miss":974,"hit_rate":37.0801,"demand_bytes":2106458112,"prefetch_bytes":0},{"layer":28,"req":1548,"hit":579,"miss":969,"hit_rate":37.4031,"demand_bytes":1619361792,"prefetch_bytes":0},{"layer":29,"req":1548,"hit":708,"miss":840,"hit_rate":45.7364,"demand_bytes":1403781120,"prefetch_bytes":0},{"layer":30,"req":1548,"hit":649,"miss":899,"hit_rate":41.9251,"demand_bytes":1502380032,"prefetch_bytes":0},{"layer":31,"req":1548,"hit":574,"miss":974,"hit_rate":37.0801,"demand_bytes":1627717632,"prefetch_bytes":0},{"layer":32,"req":1548,"hit":658,"miss":890,"hit_rate":42.5065,"demand_bytes":1924792320,"prefetch_bytes":0},{"layer":33,"req":1548,"hit":682,"miss":866,"hit_rate":44.0568,"demand_bytes":1589116928,"prefetch_bytes":0},{"layer":34,"req":1548,"hit":658,"miss":890,"hit_rate":42.5065,"demand_bytes":1633157120,"prefetch_bytes":0},{"layer":35,"req":1548,"hit":640,"miss":908,"hit_rate":41.3437,"demand_bytes":1517420544,"prefetch_bytes":0},{"layer":36,"req":1548,"hit":690,"miss":858,"hit_rate":44.5736,"demand_bytes":1433862144,"prefetch_bytes":0},{"layer":37,"req":1548,"hit":702,"miss":846,"hit_rate":45.3488,"demand_bytes":1552416768,"prefetch_bytes":0},{"layer":38,"req":1548,"hit":667,"miss":881,"hit_rate":43.0879,"demand_bytes":1472299008,"prefetch_bytes":0},{"layer":39,"req":1548,"hit":682,"miss":866,"hit_rate":44.0568,"demand_bytes":1589116928,"prefetch_bytes":0},{"layer":40,"req":1548,"hit":682,"miss":866,"hit_rate":44.0568,"demand_bytes":1447231488,"prefetch_bytes":0},{"layer":41,"req":1548,"hit":678,"miss":870,"hit_rate":43.7984,"demand_bytes":1453916160,"prefetch_bytes":0},{"layer":42,"req":1548,"hit":611,"miss":937,"hit_rate":39.4703,"demand_bytes":1565884416,"prefetch_bytes":0},{"layer":43,"req":1548,"hit":658,"miss":890,"hit_rate":42.5065,"demand_bytes":1487339520,"prefetch_bytes":0},{"layer":44,"req":1548,"hit":532,"miss":1016,"hit_rate":34.3669,"demand_bytes":1697906688,"prefetch_bytes":0},{"layer":45,"req":1548,"hit":549,"miss":999,"hit_rate":35.4651,"demand_bytes":1669496832,"prefetch_bytes":0},{"layer":46,"req":1548,"hit":656,"miss":892,"hit_rate":42.3773,"demand_bytes":1636827136,"prefetch_bytes":0},{"layer":47,"req":1548,"hit":612,"miss":936,"hit_rate":39.5349,"demand_bytes":1564213248,"prefetch_bytes":0},{"layer":48,"req":774,"hit":307,"miss":467,"hit_rate":39.6641,"demand_bytes":780435456,"prefetch_bytes":0}]}
+{"phase":"decode","tokens":514,"expert_requests":143406,"expert_hits":55340,"expert_misses":88066,"hit_rate":38.5897,"ssb_demand_bytes":169396117504,"ssb_prefetch_bytes":0,"ssb_total_bytes":169396117504,"ssd_bytes_per_token":3.29564e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":86806,"demand_wait_us":201462808,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":3220439040,"rss_bytes":5136560128,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":541,"miss":2543,"hit_rate":17.5422,"demand_bytes":4124786688,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":546,"miss":2538,"hit_rate":17.7043,"demand_bytes":5488902144,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":716,"miss":2368,"hit_rate":23.2166,"demand_bytes":6052380672,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":761,"miss":2323,"hit_rate":24.6757,"demand_bytes":5023924224,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":853,"miss":2231,"hit_rate":27.6589,"demand_bytes":4824956928,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":1017,"miss":2067,"hit_rate":32.9767,"demand_bytes":4470276096,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":1365,"miss":1719,"hit_rate":44.2607,"demand_bytes":3957055488,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":1070,"miss":2014,"hit_rate":34.6952,"demand_bytes":4355653632,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":978,"miss":2106,"hit_rate":31.7121,"demand_bytes":4554620928,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":1124,"miss":1960,"hit_rate":36.4462,"demand_bytes":4238868480,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":1277,"miss":1807,"hit_rate":41.4073,"demand_bytes":3907977216,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":1187,"miss":1897,"hit_rate":38.489,"demand_bytes":4102619136,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":1256,"miss":1828,"hit_rate":40.7263,"demand_bytes":3953393664,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":1460,"miss":1624,"hit_rate":47.3411,"demand_bytes":3512205312,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":1412,"miss":1672,"hit_rate":45.7847,"demand_bytes":3616014336,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":1354,"miss":1730,"hit_rate":43.904,"demand_bytes":3741450240,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":1456,"miss":1628,"hit_rate":47.2114,"demand_bytes":3520856064,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":1510,"miss":1574,"hit_rate":48.9624,"demand_bytes":2630418432,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":1577,"miss":1507,"hit_rate":51.1349,"demand_bytes":2765357056,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":1345,"miss":1739,"hit_rate":43.6122,"demand_bytes":3305046016,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":1328,"miss":1756,"hit_rate":43.061,"demand_bytes":3337355264,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":1341,"miss":1743,"hit_rate":43.4825,"demand_bytes":3312648192,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":1307,"miss":1777,"hit_rate":42.38,"demand_bytes":3260809216,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":1084,"miss":2000,"hit_rate":35.1492,"demand_bytes":3670016000,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":1149,"miss":1935,"hit_rate":37.2568,"demand_bytes":3550740480,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":1126,"miss":1958,"hit_rate":36.511,"demand_bytes":4234543104,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":1113,"miss":1971,"hit_rate":36.0895,"demand_bytes":3293872128,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":1309,"miss":1775,"hit_rate":42.4449,"demand_bytes":2966323200,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":1237,"miss":1847,"hit_rate":40.1102,"demand_bytes":3086647296,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":1150,"miss":1934,"hit_rate":37.2892,"demand_bytes":3232038912,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":1319,"miss":1765,"hit_rate":42.7691,"demand_bytes":3817144320,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":1274,"miss":1810,"hit_rate":41.31,"demand_bytes":3321364480,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":1297,"miss":1787,"hit_rate":42.0558,"demand_bytes":3279159296,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":1191,"miss":1893,"hit_rate":38.6187,"demand_bytes":3163521024,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":1291,"miss":1793,"hit_rate":41.8612,"demand_bytes":2996404224,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":1283,"miss":1801,"hit_rate":41.6018,"demand_bytes":3304849408,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":1306,"miss":1778,"hit_rate":42.3476,"demand_bytes":2971336704,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":1341,"miss":1743,"hit_rate":43.4825,"demand_bytes":3198418944,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":1369,"miss":1715,"hit_rate":44.3904,"demand_bytes":2866053120,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":1289,"miss":1795,"hit_rate":41.7964,"demand_bytes":2999746560,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":1090,"miss":1994,"hit_rate":35.3437,"demand_bytes":3332308992,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":1268,"miss":1816,"hit_rate":41.1154,"demand_bytes":3034841088,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":1044,"miss":2040,"hit_rate":33.8521,"demand_bytes":3409182720,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":1019,"miss":2065,"hit_rate":33.0415,"demand_bytes":3450961920,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":1225,"miss":1859,"hit_rate":39.7211,"demand_bytes":3411279872,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":1207,"miss":1877,"hit_rate":39.1375,"demand_bytes":3136782336,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":578,"miss":964,"hit_rate":37.4838,"demand_bytes":1611005952,"prefetch_bytes":0}]}
+{"phase":"shutdown","tokens":514,"expert_requests":143406,"expert_hits":55340,"expert_misses":88066,"hit_rate":38.5897,"ssb_demand_bytes":169396117504,"ssb_prefetch_bytes":0,"ssb_total_bytes":169396117504,"ssd_bytes_per_token":3.29564e+08,"prefetch_issued":0,"prefetch_used":0,"prefetch_used_rate":0,"prefetch_bytes_used":0,"prefetch_bytes_wasted":0,"prefetch_waste_rate":0,"prefetch_duplicate":0,"evictions":86806,"demand_wait_us":201462808,"prefetch_wait_us":0,"arena_bytes":3220439040,"arena_used_bytes":3220439040,"rss_bytes":3343405056,"n_slots":1260,"n_small_slots":0,"n_big_slots":1260,"small_slot_bytes":2555904,"big_slot_bytes":2555904,"slot_bytes":2555904,"n_moe_layers":47,"expert_bytes_total":23049797632,"arch":"flint","name":"Flint Preview","per_layer":[{"layer":0,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":1,"req":0,"hit":0,"miss":0,"hit_rate":0,"demand_bytes":0,"prefetch_bytes":0},{"layer":2,"req":3084,"hit":541,"miss":2543,"hit_rate":17.5422,"demand_bytes":4124786688,"prefetch_bytes":0},{"layer":3,"req":3084,"hit":546,"miss":2538,"hit_rate":17.7043,"demand_bytes":5488902144,"prefetch_bytes":0},{"layer":4,"req":3084,"hit":716,"miss":2368,"hit_rate":23.2166,"demand_bytes":6052380672,"prefetch_bytes":0},{"layer":5,"req":3084,"hit":761,"miss":2323,"hit_rate":24.6757,"demand_bytes":5023924224,"prefetch_bytes":0},{"layer":6,"req":3084,"hit":853,"miss":2231,"hit_rate":27.6589,"demand_bytes":4824956928,"prefetch_bytes":0},{"layer":7,"req":3084,"hit":1017,"miss":2067,"hit_rate":32.9767,"demand_bytes":4470276096,"prefetch_bytes":0},{"layer":8,"req":3084,"hit":1365,"miss":1719,"hit_rate":44.2607,"demand_bytes":3957055488,"prefetch_bytes":0},{"layer":9,"req":3084,"hit":1070,"miss":2014,"hit_rate":34.6952,"demand_bytes":4355653632,"prefetch_bytes":0},{"layer":10,"req":3084,"hit":978,"miss":2106,"hit_rate":31.7121,"demand_bytes":4554620928,"prefetch_bytes":0},{"layer":11,"req":3084,"hit":1124,"miss":1960,"hit_rate":36.4462,"demand_bytes":4238868480,"prefetch_bytes":0},{"layer":12,"req":3084,"hit":1277,"miss":1807,"hit_rate":41.4073,"demand_bytes":3907977216,"prefetch_bytes":0},{"layer":13,"req":3084,"hit":1187,"miss":1897,"hit_rate":38.489,"demand_bytes":4102619136,"prefetch_bytes":0},{"layer":14,"req":3084,"hit":1256,"miss":1828,"hit_rate":40.7263,"demand_bytes":3953393664,"prefetch_bytes":0},{"layer":15,"req":3084,"hit":1460,"miss":1624,"hit_rate":47.3411,"demand_bytes":3512205312,"prefetch_bytes":0},{"layer":16,"req":3084,"hit":1412,"miss":1672,"hit_rate":45.7847,"demand_bytes":3616014336,"prefetch_bytes":0},{"layer":17,"req":3084,"hit":1354,"miss":1730,"hit_rate":43.904,"demand_bytes":3741450240,"prefetch_bytes":0},{"layer":18,"req":3084,"hit":1456,"miss":1628,"hit_rate":47.2114,"demand_bytes":3520856064,"prefetch_bytes":0},{"layer":19,"req":3084,"hit":1510,"miss":1574,"hit_rate":48.9624,"demand_bytes":2630418432,"prefetch_bytes":0},{"layer":20,"req":3084,"hit":1577,"miss":1507,"hit_rate":51.1349,"demand_bytes":2765357056,"prefetch_bytes":0},{"layer":21,"req":3084,"hit":1345,"miss":1739,"hit_rate":43.6122,"demand_bytes":3305046016,"prefetch_bytes":0},{"layer":22,"req":3084,"hit":1328,"miss":1756,"hit_rate":43.061,"demand_bytes":3337355264,"prefetch_bytes":0},{"layer":23,"req":3084,"hit":1341,"miss":1743,"hit_rate":43.4825,"demand_bytes":3312648192,"prefetch_bytes":0},{"layer":24,"req":3084,"hit":1307,"miss":1777,"hit_rate":42.38,"demand_bytes":3260809216,"prefetch_bytes":0},{"layer":25,"req":3084,"hit":1084,"miss":2000,"hit_rate":35.1492,"demand_bytes":3670016000,"prefetch_bytes":0},{"layer":26,"req":3084,"hit":1149,"miss":1935,"hit_rate":37.2568,"demand_bytes":3550740480,"prefetch_bytes":0},{"layer":27,"req":3084,"hit":1126,"miss":1958,"hit_rate":36.511,"demand_bytes":4234543104,"prefetch_bytes":0},{"layer":28,"req":3084,"hit":1113,"miss":1971,"hit_rate":36.0895,"demand_bytes":3293872128,"prefetch_bytes":0},{"layer":29,"req":3084,"hit":1309,"miss":1775,"hit_rate":42.4449,"demand_bytes":2966323200,"prefetch_bytes":0},{"layer":30,"req":3084,"hit":1237,"miss":1847,"hit_rate":40.1102,"demand_bytes":3086647296,"prefetch_bytes":0},{"layer":31,"req":3084,"hit":1150,"miss":1934,"hit_rate":37.2892,"demand_bytes":3232038912,"prefetch_bytes":0},{"layer":32,"req":3084,"hit":1319,"miss":1765,"hit_rate":42.7691,"demand_bytes":3817144320,"prefetch_bytes":0},{"layer":33,"req":3084,"hit":1274,"miss":1810,"hit_rate":41.31,"demand_bytes":3321364480,"prefetch_bytes":0},{"layer":34,"req":3084,"hit":1297,"miss":1787,"hit_rate":42.0558,"demand_bytes":3279159296,"prefetch_bytes":0},{"layer":35,"req":3084,"hit":1191,"miss":1893,"hit_rate":38.6187,"demand_bytes":3163521024,"prefetch_bytes":0},{"layer":36,"req":3084,"hit":1291,"miss":1793,"hit_rate":41.8612,"demand_bytes":2996404224,"prefetch_bytes":0},{"layer":37,"req":3084,"hit":1283,"miss":1801,"hit_rate":41.6018,"demand_bytes":3304849408,"prefetch_bytes":0},{"layer":38,"req":3084,"hit":1306,"miss":1778,"hit_rate":42.3476,"demand_bytes":2971336704,"prefetch_bytes":0},{"layer":39,"req":3084,"hit":1341,"miss":1743,"hit_rate":43.4825,"demand_bytes":3198418944,"prefetch_bytes":0},{"layer":40,"req":3084,"hit":1369,"miss":1715,"hit_rate":44.3904,"demand_bytes":2866053120,"prefetch_bytes":0},{"layer":41,"req":3084,"hit":1289,"miss":1795,"hit_rate":41.7964,"demand_bytes":2999746560,"prefetch_bytes":0},{"layer":42,"req":3084,"hit":1090,"miss":1994,"hit_rate":35.3437,"demand_bytes":3332308992,"prefetch_bytes":0},{"layer":43,"req":3084,"hit":1268,"miss":1816,"hit_rate":41.1154,"demand_bytes":3034841088,"prefetch_bytes":0},{"layer":44,"req":3084,"hit":1044,"miss":2040,"hit_rate":33.8521,"demand_bytes":3409182720,"prefetch_bytes":0},{"layer":45,"req":3084,"hit":1019,"miss":2065,"hit_rate":33.0415,"demand_bytes":3450961920,"prefetch_bytes":0},{"layer":46,"req":3084,"hit":1225,"miss":1859,"hit_rate":39.7211,"demand_bytes":3411279872,"prefetch_bytes":0},{"layer":47,"req":3084,"hit":1207,"miss":1877,"hit_rate":39.1375,"demand_bytes":3136782336,"prefetch_bytes":0},{"layer":48,"req":1542,"hit":578,"miss":964,"hit_rate":37.4838,"demand_bytes":1611005952,"prefetch_bytes":0}]}

results/hot/test_lru_1024.log

@@ -0,0 +1,29 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=420  arena=1.00 GiB  (RAM 實測 RSS=1.05 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 420
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.01.021.433 I cmn          init: llama threadpool init, n_threads = 16
+0.01.530.654 I 
+0.01.530.743 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.530.764 I perplexity: tokenizing the input ..
+0.01.539.270 I perplexity: tokenization took 8.503 ms
+0.01.539.288 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+0.54.024.763 I perplexity: 52.49 seconds per pass - ETA 1.73 minutes
+[1]9.4981,[2]6.4793,
+1.48.168.299 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 2 次，mode C 23899 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=420  arena=1.00 GiB  (RAM 實測 RSS=1.12 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 420
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_lru_128.log

@@ -0,0 +1,29 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=52  arena=0.12 GiB  (RAM 實測 RSS=0.17 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 52
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.00.565.207 I cmn          init: llama threadpool init, n_threads = 16
+0.01.021.710 I 
+0.01.021.809 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.021.829 I perplexity: tokenizing the input ..
+0.01.027.706 I perplexity: tokenization took 5.872 ms
+0.01.027.723 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+1.05.178.496 I perplexity: 64.15 seconds per pass - ETA 2.13 minutes
+[1]9.4981,[2]6.4793,
+2.12.154.212 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 23901 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=52  arena=0.12 GiB  (RAM 實測 RSS=0.24 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 52
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_lru_256.log

@@ -0,0 +1,29 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.30 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.00.639.503 I cmn          init: llama threadpool init, n_threads = 16
+0.01.127.464 I 
+0.01.127.566 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.127.588 I perplexity: tokenizing the input ..
+0.01.133.980 I perplexity: tokenization took 6.388 ms
+0.01.133.995 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+1.07.327.951 I perplexity: 66.19 seconds per pass - ETA 2.20 minutes
+[1]9.4981,[2]6.4793,
+2.14.112.373 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 23901 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.37 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_lru_3072.log

@@ -0,0 +1,29 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=1260  arena=3.00 GiB  (RAM 實測 RSS=3.05 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 1260
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.02.318.215 I cmn          init: llama threadpool init, n_threads = 16
+0.02.831.768 I 
+0.02.831.872 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.02.831.892 I perplexity: tokenizing the input ..
+0.02.836.895 I perplexity: tokenization took 5.001 ms
+0.02.836.912 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+0.48.992.475 I perplexity: 46.16 seconds per pass - ETA 1.53 minutes
+[1]9.4981,[2]6.4793,
+1.33.517.629 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 235 次，mode C 23666 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=1260  arena=3.00 GiB  (RAM 實測 RSS=3.11 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 1260
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_pin_1024.log

@@ -0,0 +1,30 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq] expert 剖面：/tmp/opencode/flint-work/hot/profile.bin → 釘選 105 / 11817 個熱門 expert（永不淘汰）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：105 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=420  arena=1.00 GiB  (RAM 實測 RSS=1.05 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 420
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.01.102.128 I cmn          init: llama threadpool init, n_threads = 16
+0.01.612.214 I 
+0.01.612.321 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.612.347 I perplexity: tokenizing the input ..
+0.01.620.270 I perplexity: tokenization took 7.919 ms
+0.01.620.290 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+0.53.690.944 I perplexity: 52.07 seconds per pass - ETA 1.73 minutes
+[1]9.4981,[2]6.4793,
+1.46.362.924 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 5 次，mode C 23896 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：105 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=420  arena=1.00 GiB  (RAM 實測 RSS=1.12 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 420
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_pin_128.log

@@ -0,0 +1,30 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq] expert 剖面：/tmp/opencode/flint-work/hot/profile.bin → 釘選 13 / 11817 個熱門 expert（永不淘汰）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：13 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=52  arena=0.12 GiB  (RAM 實測 RSS=0.17 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 52
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.00.553.909 I cmn          init: llama threadpool init, n_threads = 16
+0.00.981.131 I 
+0.00.981.244 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.00.981.268 I perplexity: tokenizing the input ..
+0.00.987.922 I perplexity: tokenization took 6.648 ms
+0.00.987.942 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+1.03.973.963 I perplexity: 62.99 seconds per pass - ETA 2.08 minutes
+[1]9.4981,[2]6.4793,
+2.08.423.305 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 23901 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：13 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=52  arena=0.12 GiB  (RAM 實測 RSS=0.24 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 52
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_pin_256.log

@@ -0,0 +1,30 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq] expert 剖面：/tmp/opencode/flint-work/hot/profile.bin → 釘選 26 / 11817 個熱門 expert（永不淘汰）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：26 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.30 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.00.604.751 I cmn          init: llama threadpool init, n_threads = 16
+0.01.135.678 I 
+0.01.135.773 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.01.135.792 I perplexity: tokenizing the input ..
+0.01.140.580 I perplexity: tokenization took 4.784 ms
+0.01.140.594 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+1.06.000.523 I perplexity: 64.86 seconds per pass - ETA 2.15 minutes
+[1]9.4981,[2]6.4793,
+2.13.160.194 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 23901 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：26 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=105  arena=0.25 GiB  (RAM 實測 RSS=0.36 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 105
+[sdq]   prefetch=off  fadvise_dontneed=1

results/hot/test_pin_3072.log

@@ -0,0 +1,30 @@
+[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert
+[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B）
+[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache）
+[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊）
+[sdq] expert 剖面：/tmp/opencode/flint-work/hot/profile.bin → 釘選 315 / 11817 個熱門 expert（永不淘汰）
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：315 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=1260  arena=3.00 GiB  (RAM 實測 RSS=3.05 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 1260
+[sdq]   prefetch=off  fadvise_dontneed=1
+0.02.267.902 I cmn          init: llama threadpool init, n_threads = 16
+0.02.753.297 I 
+0.02.753.395 I system_info: n_threads = 16 (n_threads_batch = 16) / 192 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
+0.02.753.416 I perplexity: tokenizing the input ..
+0.02.758.617 I perplexity: tokenization took 5.198 ms
+0.02.758.632 I perplexity: calculating perplexity over 2 chunks, n_ctx=256, batch_size=256, n_seq=1
+0.49.806.537 I perplexity: 47.05 seconds per pass - ETA 1.57 minutes
+[1]9.4981,[2]6.4793,
+1.38.315.559 I Final estimate: PPL = 6.4793 +/- 1.24611
+
+[sdq] MoE op 路徑計數：row-parallel 242 次，mode C 23659 次
+[sdq]   page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048
+[sdq] expert pager ready: models/flint-UD-Q4_K_XL.gguf
+[sdq]   熱門 expert 釘選：315 個（永不淘汰）
+[sdq]   slot=2.44 MiB (2555904 B)  slots=1260  arena=3.00 GiB  (RAM 實測 RSS=3.11 GiB)
+[sdq]   evict_policy=lru  stale_window=640
+[sdq]   槽位：單一尺寸 2.44 MiB × 1260
+[sdq]   prefetch=off  fadvise_dontneed=1

results/test-status.md

@@ -2,7 +2,7 @@
 
 Environment: Intel Xeon Platinum 8559C, container quota 16 CPUs, cgroup v2 `memory.max` 104,000,000,000 bytes (~96.9 GiB), `memory.swap.max` 2,936,848,384 bytes (~2.74 GiB), Amazon Linux 2023, kernel 6.18.48, CPU only. The container's cgroup v2 mount is read-only and there is no systemd manager, so no cgroup-enforced memory limit could be applied.
 
-llama.cpp base: `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` plus `flint-epq.patch` (sha256 `b8a4e77af3743bd35bd4c14258c70c12031e60ba264f95ab93593519ea0ab08c`).
+llama.cpp base: `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` plus `flint-epq.patch` (sha256 `94559012e2ea65cad6ce2175927c19095d13256bb4ec8fb2fdcd163570e00385`).
 
 Versions: **native** = `SDQ_PAGER=0`. **v1** = expert paging, mode C (`SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`). **v2** = v1 + preload + row-parallel for gated experts (`SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1`, arena ≥ 28.6 GiB). **v2 forced** = v2 with `SDQ_ROWPAR_MAX_MISS=100000` (ablation).
 
@@ -29,7 +29,7 @@ Versions: **native** = `SDQ_PAGER=0`. **v1** = expert paging, mode C (`SDQ_PRELO
 | Row-parallel active check | v2 | `rowpar=1` on decode ops at 16 threads; `rowpar=0` with `SDQ_ROW_PARALLEL=0` |
 | Path counter (`MoE op 路徑計數`) | v2 | 425 row-parallel / 91 mode C at 32 GiB; 0 / 516 at 2–8 GiB; 12 / 504 at 16 GiB |
 | Preload check | v2 | "已預載全部 expert（12061 slots）" with a 28.7 GiB arena |
-| Fresh checkout of patch (sha `b8a4e77…`): build, counter reproduced | v2 | PASS: 425 / 91 reproduced (run before the watchdog fix; patch re-verified to apply cleanly) |
+| Fresh checkout of patch (sha `94559012…`): build, counter reproduced | v2 | PASS: 425 / 91 reproduced (run before the watchdog fix; patch re-verified to apply cleanly) |
 | Throughput, context 512, batch 128, ubatch 32, 32 tokens, 16 threads (3 runs, median) | native / v1 / v2 | Generation 21.5 / 5.6 / 16.2 t/s; prompt 54.4 / 13.0 / 32.4 t/s. Peak RSS 24.3 / 19.8 / 30.5 GiB. No watchdog applied |
 | Memory tiers, **enforced**, native, 2 / 4 / 8 / 16 / 32 GiB, 16 threads (3 runs each) | native | 2, 4, 8, 16 GiB: killed in 3 / 3 runs each (peaks 2.2–3.0, 4.1–4.3, 8.1–8.2, 16.0–16.5 GiB at kill). 32 GiB: pass, peak 24.286 GiB, generation 21.8 t/s (median) |
 | Memory tiers, **enforced**, v1, 2 / 4 / 8 / 16 / 32 GiB, 16 threads (3 runs each) | v1 | 2 GiB with 256 MiB arena: killed (peak 2.000–2.001 GiB). 4 / 8 / 16 / 32 GiB: pass, peaks 2.765 / 4.766 / 11.766 / 19.765 GiB |
@@ -37,6 +37,12 @@ Versions: **native** = `SDQ_PAGER=0`. **v1** = expert paging, mode C (`SDQ_PRELO
 | Memory tiers, **enforced**, v2 as designed, 2 / 4 / 8 / 16 / 32 GiB | v2 | 2 GiB (256 MiB): killed. 4 / 8 / 16 GiB: pass, peaks 2.765 / 4.765 / 11.766 GiB, 0 / 0 / 12 RP ops. 32 GiB (28.7 GiB arena): pass, peak 30.473 GiB, 425 RP ops |
 | Memory tiers, **enforced**, v2 forced, 2 / 4 / 8 / 16 / 32 GiB | v2 forced | 2 GiB (256 MiB): killed. 4 / 8 / 16 / 32 GiB: pass, peaks 2.759 / 4.759 / 11.760 / 19.759 GiB, 425 RP ops each |
 | cgroup-v2 `MemoryMax` runs (`tools/runmem_cgroup.py`) | — | NOT RUN: no writable cgroup v2 / systemd here. The script is untested |
+| Expert routing skew (`SDQ_ROUTE_LOG`, 1,017 tokens per corpus) | — | Gini 0.38–0.62 per layer; top 6 % of experts = 19–36 % of requests; profile-to-test rank correlation 0.59 |
+| Hot-profile hit counting (fixed: hits now counted) | v1 | profile records 286,254 requests (1,017 tokens × 47 layers × 6 = 286,794 minus unused) |
+| Profile-pinned vs LRU hit rate, 514 tokens, 128 / 256 / 1,024 / 3,072 MiB | v1 | LRU / pinned: 0 / 0, 0 / 0, 19.3 / 21.5, 40.3 / 37.5 % |
+| Perplexity identical with and without pinning, all four arenas | v1 | 6.4793 in all runs (caching changes speed only) |
+| Fresh checkout of patch (sha `94559012…`): applies and builds | v1, v2 | PASS |
+
 
 ## Defects found and fixed
 
@@ -44,7 +50,7 @@ Versions: **native** = `SDQ_PAGER=0`. **v1** = expert paging, mode C (`SDQ_PRELO
 2. **`swiglu_limit` was dropped:** `sdq_build_moe` ignored its argument and the op always used 0. Fixed. This checkpoint has limit 0, so the numbers are unaffected.
 3. **SwiGLU clamp mismatch:** the pager clamped `silu(gate)` to `[-limit, limit]`; upstream clamps only the upper bound. Fixed to match upstream.
 4. **Bad debug hook (debug path only):** `sdq_maybe_dump_tensor` (enabled by `SDQ_MOE_OUT_DUMP`) wrote the tensor to a file but never copied it into its output, so downstream nodes read uninitialized memory. Fixed; the production path never used this node.
-5. **Published patch was stale** at several points. Re-generated and re-uploaded after each change; the current patch is `b8a4e77…`.
+5. **Published patch was stale** at several points. Re-generated and re-uploaded after each change; the current patch is `94559012…`.
 6. **Smoke command in the model card was not the one that was run.** The card shows the executed command.
 7. **`tools/runmem_cgroup.py` and the cgroup claim were presented as usable.** Labeled untested.
 8. **Earlier memory-tier throughput came from single runs** and varied by ~±15 %. Replaced with medians of 3 runs.
@@ -52,6 +58,8 @@ Versions: **native** = `SDQ_PAGER=0`. **v1** = expert paging, mode C (`SDQ_PRELO
 10. **Row-parallel was inactive for Flint.** Added the gated path; verified active and numerically equivalent to v1.
 11. **The forced-row-parallel speed gain (9–14 %) was not reproducible.** It came from the unenforced runs. With enforced runs the medians are within noise of v1 and v2 at all tiers. Retracted.
 
+12. **Hot-expert profile counted only misses, and the default pinned everything.** Hits are now counted, and the default pinned set is 25 % of arena slots (used only when a profile is given). Measured effect is small: +2.2 points at 1 GiB, −2.8 at 3 GiB, zero below 256 MiB.
+
 ## Known limits and open items
 
 - EPQ output is not bit-identical to native llama.cpp. Float summation order differs, and 8-bit activation quantization amplifies the difference through the layers. Native itself shows a KL divergence of the same magnitude under a ubatch change. No functional mismatch was found.

src/sdq_pager.cpp

@@ -1551,7 +1551,10 @@ bool pager::load_profile(const std::string & path) {
               [](const auto & a, const auto & b) { return a.first > b.first; });
 
     hot_.assign((size_t) info_.n_layer * (size_t) info_.n_expert, 0);
-    const int want = cfg_.hot_pin > 0 ? cfg_.hot_pin : (int) ranked.size();
+    // 預設釘選 25 % 的槽位（模擬顯示這是 1 GiB 階層的最佳點）。舊版沒有設定時
+    // 會把所有出現過的 expert 都標成熱門，等於沒有冷槽可淘汰。
+    const int want = cfg_.hot_pin > 0 ? cfg_.hot_pin
+                                      : std::max<int>(1, (int) (n_slots_ / 4));
     n_hot_ = 0;
     for (const auto & r : ranked) {
         if (n_hot_ >= want) {
@@ -1737,6 +1740,9 @@ int pager::acquire(int il, int ie, bool blocking) {
         slots_[s].last_use = ++clock_;
         slots_[s].pin++;
         touch_freq_locked(slots_[(size_t) s]);
+        // 剖面必須統計「所有」請求（命中也要算）。舊版只在 miss 路徑計數，
+        // 熱門 expert 因為大多命中，會被剖面系統性低估。
+        count_request(il, ie);
         stats_.expert_requests.fetch_add(1, std::memory_order_relaxed);
         stats_.expert_hits.fetch_add(1, std::memory_order_relaxed);
         if (lstats_ && il >= 0 && il < (int) info_.n_layer) {