diff --git a/README.md b/README.md index 4e0e92b09cdac0ec855034436b948a229bbc0d40..d42a935318f43a253cf318f5f592ae64ec25e5e2 100644 --- a/README.md +++ b/README.md @@ -14,12 +14,13 @@ tags: CPU expert paging for the Unsloth `flint-UD-Q4_K_XL.gguf` checkpoint. Routed gate/up/down expert weights stay on disk and are copied into a bounded RAM arena as needed. Flint's router, selected-expert weights, shared expert, attention and dense layers remain on the upstream llama.cpp graph. -Two versions are measured throughout: +Three configurations are measured throughout: +- **native**: unmodified llama.cpp MoE path (`SDQ_PAGER=0`), the reference. - **v1**: expert paging with the per-expert custom op ("mode C"). `SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`. -- **v2**: v1 plus preload of all experts into the arena and row-parallel decode for gated (gate/up/down) experts. `SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1`, arena ≥ 28.6 GiB. - -Native llama.cpp (`SDQ_PAGER=0`) is the reference. +- **v2**: v1 plus preload of all experts into the arena and row-parallel decode for gated (gate/up/down) experts. `SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1`, arena ≥ 28.6 GiB. Two sub-variants: + - **v2 as designed**: row-parallel runs only when every needed expert is resident (default guard). + - **v2 forced**: `SDQ_ROWPAR_MAX_MISS=100000`, row-parallel runs even when some needed experts are missing (ablation; see below). ## Source model @@ -35,17 +36,17 @@ System information recorded on 2026-10-23: |---|---| | CPU | Intel Xeon Platinum 8559C, 2 sockets × 48 cores, SMT enabled (192 host logical CPUs) | | CPU features | AVX2, AVX-512, VNNI, BF16; AMX available on host but disabled for this build | -| Container CPU quota | 16 CPUs (`cpu.max` = 1,600,000 / 100,000); v1 memory tiers used 16 threads, v2 and the throughput runs used 16 threads | +| Container CPU quota | 16 CPUs (`cpu.max` = 1,600,000 / 100,000). All memory and throughput runs used 16 threads | | Host memory | 2.0 TiB reported by `free` | | Container memory limit | 104,000,000,000 bytes (about 96.9 GiB), cgroup v2 | | Container swap limit | 2,936,848,384 bytes (about 2.74 GiB); measured test processes used 0 swap | | GPU | No GPU used; CPU-only build and inference | | OS / kernel | Amazon Linux 2023 x86_64, Linux 6.18.48 | -| llama.cpp | `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` (`970b51c`) + `flint-epq.patch` (sha256 `f6ae3fd33dcd29a132e37bfebacc6c452778e752266bb963718f8ec6f9522572`) | +| llama.cpp | `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` (`970b51c`) + `flint-epq.patch` (sha256 `b8a4e77af3743bd35bd4c14258c70c12031e60ba264f95ab93593519ea0ab08c`) | ## Build -Start from llama.cpp commit `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc`, apply `flint-epq.patch`, then build CPU tools. This exact sequence was verified on a fresh checkout. +Start from llama.cpp commit `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc`, apply `flint-epq.patch`, then build CPU tools. The patch was verified on a fresh checkout. ```bash git clone https://github.com/ggml-org/llama.cpp @@ -77,7 +78,7 @@ SDQ_PAGER=1 SDQ_PREFETCH=0 SDQ_ARENA_MB=29400 SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1 \ -p 'Explain in one sentence why the sky is blue.' -n 32 -t 16 --temp 0 --single-turn --simple-io ``` -`SDQ_PAGER=0` selects the native llama.cpp MoE path. `SDQ_ARENA_MB` sets the expert cache size. `SDQ_STATS_FILE=/path/to/stats.jsonl` writes JSON statistics. +`SDQ_PAGER=0` selects the native llama.cpp MoE path. `SDQ_ARENA_MB` sets the expert cache size. `SDQ_STATS_FILE=/path/to/stats.jsonl` writes JSON statistics. At exit the program prints `MoE op 路徑計數：row-parallel N 次，mode C M 次`, which shows which path each MoE op used. ## Numerical validation @@ -93,60 +94,72 @@ Reference: native llama.cpp (`SDQ_PAGER=0`) on the same binary. Text: the first | Decode KL vs native (`-ub 1`, 16 threads, 2 chunks) | — | 0.044057; top-1 92.5 % | 0.044057; top-1 92.5 % | | **Control:** native `-ub 64` vs native `-ub 1` (only float order changes) | — | — | mean KLD 0.034464; top-1 90.2 % | | Decode v2 vs v1 (`-ub 1`, 16 threads) | — | — | KLD 1e-6 (max 3.8e-5); top-1 100 % | +| Decode, v2 forced with 1 GiB arena vs native | — | — | 0.044057; identical to v1 at reported precision | | Layer-wise MoE output, layers 2–6, EPQ vs native (same input) | — | max relative diff ≤ 1.6e-5 | — | Findings: - **v2 row-parallel is numerically equivalent to v1** in decode (KL 1e-6, top-1 100 %) and identical to v1 in prefill (prefill does not use row-parallel). It is not bit-identical. -- **EPQ is not bit-identical to native.** Float summation order differs, and 8-bit activation quantization amplifies the difference through the layers. The native `-ub 1` vs `-ub 64` control gives a KL divergence of the same magnitude (0.034 vs 0.044). The deviation is in the range of ordinary reordering sensitivity for this checkpoint; it is a measured conclusion from these runs, not a proof of exact equivalence. -- **Greedy text identity is fragile.** At 8 threads the 24-token output matches native exactly; at 16 threads it diverges after 12 characters, for v1 and v2 alike. The divergence is a near-tie argmax flip caused by the reordering above. +- **EPQ is not bit-identical to native.** Float summation order differs, and 8-bit activation quantization amplifies the difference through the layers. The native `-ub 1` vs `-ub 64` control gives a KL divergence of the same magnitude (0.034 vs 0.044). This is a measured conclusion from these runs, not a proof of exact equivalence. +- **Greedy text identity is fragile.** At 8 threads the 24-token output matches native exactly; at 16 threads it diverges after 12 characters, for v1 and v2 alike. -## Memory benchmark: v1 and v2 at each tier +## Memory benchmark (RSS watchdog, enforced) -`bench.sh ` runs a fixed prompt and samples process-tree RSS every 20 ms. If sampled RSS exceeds the requested limit, the watchdog kills the run. This is a sampled RSS watchdog, not a kernel/cgroup hard limit: RSS may briefly exceed the threshold between samples, and it counts file-backed pages as reported by `/proc`. `tools/runmem_cgroup.py` (cgroup-v2 `MemoryMax`) is included but **untested in this environment**; it requires a host with writable cgroup v2 and systemd. +`bench.sh ` runs a fixed prompt and samples process-tree RSS every 20 ms. If sampled RSS exceeds the tier, the watchdog kills the run (exit −9). This is a sampled watchdog, not a kernel/cgroup hard limit: RSS may briefly exceed the threshold between samples, and it counts file-backed pages as reported by `/proc`. `tools/runmem_cgroup.py` (cgroup-v2 `MemoryMax`) is included but **untested in this environment**. -Version is selected with environment variables: +> **Correction.** The threshold check in `tools/runmem.py` compared `VmRSS` in kB against GiB in bytes, so the watchdog never fired. Every "PASS, under threshold" statement in earlier versions of this card was therefore unverified. The peak RSS values were measured correctly. The check is fixed in this patch, and all tier results below are re-run with enforcement. The earlier tables are in `results/logs/superseded_runmem_threshold_bug/`. -- **v1**: `SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0` (the `bench.sh` defaults) -- **v2 as designed**: `SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1` -- **v2 forced (ablation)**: as v2, plus `SDQ_ROWPAR_MAX_MISS=100000`, which lets row-parallel run even when some needed experts are not resident +Settings: context 128, batch 32, ubatch 16, 8 generated tokens, 16 threads, prefetch off. Each row is the median of 3 runs. "killed" means the watchdog terminated all 3 runs. Path counts are RP (row-parallel) / mode C ops. -```bash -for cap in 2 4 8 16 32; do ./bench.sh "$cap" 1; done # v1, 16 threads -SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1 ARENA_MB=29400 ./bench.sh 32 1 # v2 at 32 GiB -``` +Native llama.cpp needs about 24.3 GiB, so it cannot run below 32 GiB under this watchdog. For 2 GiB, the 256 MiB arena from the earlier table sits right at the threshold (peak 2.000–2.001 GiB) and is killed, so the table adds a 128 MiB arena row that passes. + +| Tier | Configuration | Arena (slots) | Result | Peak RSS | Prompt | Generation | RP / mode C | +|---:|---|---:|---|---:|---:|---:|---:| +| 2 GiB | native | — | killed (peak 2.2–3.0 GiB at kill) | — | — | — | — | +| 2 GiB | v1 | 256 MiB (105) | killed | 2.000–2.001 GiB at kill | — | — | 0 / 516 | +| 2 GiB | v2 as designed | 256 MiB (105) | killed | 2.000–2.001 GiB at kill | — | — | 0 / 516 | +| 2 GiB | v2 forced | 256 MiB (105) | killed | 2.001 GiB at kill | — | — | 425 / 91 | +| 2 GiB | v1 | 128 MiB (52) | pass | 1.887 GiB | 6.3 t/s | 3.9 t/s | 0 / 516 | +| 2 GiB | v2 as designed | 128 MiB (52) | pass | 1.886 GiB | 6.1 t/s | 3.8 t/s | 0 / 516 | +| 2 GiB | v2 forced | 128 MiB (52) | pass | 1.883 GiB | 6.1 t/s | 3.9 t/s | 425 / 91 | +| 4 GiB | native | — | killed (peak 4.1–4.3 GiB at kill) | — | — | — | — | +| 4 GiB | v1 | 1 GiB (420) | pass | 2.765 GiB | 6.5 t/s | 4.7 t/s | 0 / 516 | +| 4 GiB | v2 as designed | 1 GiB (420) | pass | 2.765 GiB | 7.1 t/s | 4.8 t/s | 0 / 516 | +| 4 GiB | v2 forced | 1 GiB (420) | pass | 2.759 GiB | 7.0 t/s | 4.7 t/s | 425 / 91 | +| 8 GiB | native | — | killed (peak 8.1–8.2 GiB at kill) | — | — | — | — | +| 8 GiB | v1 | 3 GiB (1,260) | pass | 4.766 GiB | 8.8 t/s | 4.8 t/s | 0 / 516 | +| 8 GiB | v2 as designed | 3 GiB (1,260) | pass | 4.765 GiB | 9.3 t/s | 4.9 t/s | 0 / 516 | +| 8 GiB | v2 forced | 3 GiB (1,260) | pass | 4.759 GiB | 9.1 t/s | 5.3 t/s | 425 / 91 | +| 16 GiB | native | — | killed (peak 16.0–16.5 GiB at kill) | — | — | — | — | +| 16 GiB | v1 | 10 GiB (4,201) | pass | 11.766 GiB | 11.2 t/s | 5.5 t/s | 0 / 516 | +| 16 GiB | v2 as designed | 10 GiB (4,201) | pass | 11.766 GiB | 12.1 t/s | 5.5 t/s | 12 / 504 | +| 16 GiB | v2 forced | 10 GiB (4,201) | pass | 11.760 GiB | 12.6 t/s | 5.5 t/s | 425 / 91 | +| 32 GiB | native | — | pass | 24.286 GiB | 50.3 t/s | 21.8 t/s | — | +| 32 GiB | v1 | 18 GiB (7,561 of 12,032) | pass | 19.765 GiB | 11.3 t/s | 5.7 t/s | 0 / 516 | +| 32 GiB | v2 as designed | 28.7 GiB (12,061, all experts) | pass | 30.473 GiB | 34.7 t/s | 14.6 t/s | 425 / 91 | +| 32 GiB | v2 forced | 18 GiB (7,561 of 12,032) | pass | 19.759 GiB | 12.7 t/s | 5.2 t/s | 425 / 91 | + +Throughput is noisy: single runs vary by about ±15 %, and one v1 4 GiB run produced 2.6 t/s. Read the speed columns as approximate. + +### What the enforced results show -Workload: context 128, batch 32, ubatch 16, 8 generated tokens, 16 threads, prefetch off. Each row is the median of 3 runs, with all runs exited 0, swap 0 and below their threshold. The path counter (`MoE op 路徑計數`) is printed at exit and shows how many MoE ops used row-parallel (RP) and mode C. +- **Native llama.cpp is killed at 2, 4, 8 and 16 GiB.** It runs only with at least 24.3 GiB. EPQ is the only one of the two that runs under the lower tiers. +- **At the 2 GiB tier the expert arena must be about 128 MiB.** With 256 MiB the process peaks at 2.000–2.001 GiB and is killed. +- **v2 as designed differs from v1 only at 32 GiB with the full arena.** At 2–8 GiB no layer has all its experts resident, so v2 runs mode C (0 RP ops). At 16 GiB, 12 of 516 ops were row-parallel, which makes no measurable difference. At 32 GiB, v2 generates 14.6 t/s against 5.7 t/s for v1 (about 2.6×). +- **Forced row-parallel shows no reliable speed gain.** Its medians are within run-to-run noise of v1 and v2 at every tier. The +9–14 % gains reported in the previous version of this card came from the unenforced runs and are retracted. Output is numerically unchanged (decode KL 0.044057 at a 1 GiB forced arena). -| Tier | Version | Arena (slots) | Peak RSS | Prompt | Generation | RP / mode C ops | -|---:|---|---:|---:|---:|---:|---:| -| 2 GiB | v1 | 256 MiB (105) | 2.014 GiB | 6.9 t/s | 3.9 t/s | 0 / 516 | -| 2 GiB | v2 as designed | 256 MiB (105) | 2.014 GiB | 6.9 t/s | 3.9 t/s | 0 / 516 | -| 2 GiB | v2 forced | 256 MiB (105) | 2.009 GiB | 6.9 t/s | 3.9 t/s | 425 / 91 | -| 4 GiB | v1 | 1 GiB (420) | 2.765 GiB | 7.1 t/s | 4.4 t/s | 0 / 516 | -| 4 GiB | v2 as designed | 1 GiB (420) | 2.765 GiB | 7.1 t/s | 4.6 t/s | 0 / 516 | -| 4 GiB | v2 forced | 1 GiB (420) | 2.759 GiB | 7.2 t/s | 4.8 t/s | 425 / 91 | -| 8 GiB | v1 | 3 GiB (1,260) | 4.766 GiB | 9.4 t/s | 4.4 t/s | 0 / 516 | -| 8 GiB | v2 as designed | 3 GiB (1,260) | 4.766 GiB | 9.6 t/s | 4.7 t/s | 0 / 516 | -| 8 GiB | v2 forced | 3 GiB (1,260) | 4.759 GiB | 9.6 t/s | 5.0 t/s | 425 / 91 | -| 16 GiB | v1 | 10 GiB (4,201) | 11.767 GiB | 12.5 t/s | 5.2 t/s | 0 / 516 | -| 16 GiB | v2 as designed | 10 GiB (4,201) | 11.767 GiB | 13.0 t/s | 5.3 t/s | 12 / 504 | -| 16 GiB | v2 forced | 10 GiB (4,201) | 11.760 GiB | 12.6 t/s | 5.3 t/s | 425 / 91 | -| 32 GiB | v1 | 18 GiB (7,561 of 12,032) | 19.766 GiB | 12.0 t/s | 5.2 t/s | 0 / 516 | -| 32 GiB | v2 as designed | 28.7 GiB (12,061, all experts) | 30.473 GiB | 35.0 t/s | 14.2 t/s | 425 / 91 | -| 32 GiB | v2 forced | 18 GiB (7,561 of 12,032) | 19.759 GiB | 11.1 t/s | 5.8 t/s | 425 / 91 | +## Forced vs as designed -What this shows: +Row-parallel decode computes all experts for a layer in one pass across all threads. That is only efficient when every expert it needs is already in the arena: -- **v2 as designed only activates at the 32 GiB tier with a full arena.** The preload and the row-parallel guard need every needed expert resident. At 2–8 GiB no layer ever has all its experts resident, so v2 runs the same mode C path as v1 (0 RP ops). The 4 and 8 GiB rows differ from v1 only by run-to-run noise (about ±10 %). At 16 GiB, 12 of 516 ops happened to find their experts resident and took the row-parallel path; the effect on speed is within noise. -- **v2 at 32 GiB with the full arena: generation 14.2 t/s vs 5.2 t/s for v1 at the same tier (about 2.7×).** Peak RSS 30.5 GiB, under the 32 GiB watchdog. This is the only v2 configuration that the design targets. -- **Forced row-parallel (ablation):** generation rises about 9–14 % at 4, 8 and 32 GiB (4.8, 5.0, 5.8 t/s vs 4.4, 4.4, 5.2). These gains are at the edge of the run-to-run noise and need more runs before they are claimed. Output is numerically unchanged: decode-mode KL at a 1 GiB forced arena is 0.044057, identical at reported precision to v1 with a full arena (see `results/logs/v2/kl_decode_ub1_t16_forced_rowpar_1g_vs_native.log`). +- **As designed** (`SDQ_ROWPAR_MAX_MISS=0`, the default). A layer uses row-parallel only if every needed expert is resident. Otherwise it falls back to mode C, which reads each expert from SSD while computing the previous one, so I/O and compute overlap. +- **Forced** (`SDQ_ROWPAR_MAX_MISS=100000`). A decode op uses row-parallel whenever the number of needed experts is below the thread count, even if some experts are missing. Missing experts are then loaded with blocking reads inside row-parallel setup before compute starts, so I/O and compute no longer overlap. -Raw logs: v1 `results/logs/v1/mem_v1_*.log`; v2 `results/logs/v2/mem_default_*.log` and `results/logs/v2/mem_forced_*.log`; summaries `results/logs/common/v2_mem_summary.txt` and `results/logs/common/mem_summary.txt`. The earlier `results/logs/v2/v2_32g_run*.log` are the same 32 GiB configuration from an earlier command and are superseded by `mem_default_32g_*`. +The forced mode trades that overlap for a larger parallel region. In these runs the trade is neutral: the forced variant does 425 row-parallel ops at every tier and gives the same speed within noise. It is an ablation, not a recommended setting. A workload with many misses could be slower than mode C, which this card does not measure. -## Throughput (v1, v2, native) +## Throughput (native, v1, v2) -Separate workload from the memory tiers: context 512, batch 128, ubatch 32, 32 generated tokens, 16 threads, 3 runs each (median). +Separate workload from the memory tiers: context 512, batch 128, ubatch 32, 32 generated tokens, 16 threads, 3 runs each (median). No watchdog was applied to these runs. | Configuration | Prompt | Generation | Peak RSS | Logs | |---|---:|---:|---:|---| @@ -155,7 +168,7 @@ Separate workload from the memory tiers: context 512, batch 128, ubatch 32, 32 g | v1, arena 28.7 GiB, no preload | 12.3 t/s | 5.5 t/s | 30.51 GiB | `results/logs/v1/perf_c512_v1_arena28g_*` | | **v2**, arena 28.7 GiB, preload + row-parallel | **32.4 t/s** | **16.2 t/s** | 30.50 GiB | `results/logs/v2/perf_c512_v2_arena28g_*` | -v2 generates about 2.9× faster than v1 and about 75 % of native generation speed. The prompt phase is also faster in v2 (likely because the preload removes SSD reads during prefill, which still uses mode C; not separately measured). The v1 and v2 rows at 28.7 GiB have nearly the same RSS because the prompt already touches most experts in both runs. +v2 generates about 2.9× faster than v1 and about 75 % of native generation speed in this workload. The prompt phase is also faster in v2, likely because the preload removes SSD reads during prefill; this was not measured separately. ## Version coverage diff --git a/flint-epq.patch b/flint-epq.patch index 3ad2d723bce6af2611297db7e8056cc58e0d06bd..2d533e88e358f73819e5b2f71b9798ff16e68fb1 100644 --- a/flint-epq.patch +++ b/flint-epq.patch @@ -4688,10 +4688,10 @@ index 0000000..b109f33 \ No newline at end of file diff --git a/tools/runmem.py b/tools/runmem.py new file mode 100644 -index 0000000..765be84 +index 0000000..9d5e96c --- /dev/null +++ b/tools/runmem.py -@@ -0,0 +1,79 @@ +@@ -0,0 +1,80 @@ +#!/usr/bin/env python3 +"""Sample process-tree RSS/swap while running a command.""" + @@ -4746,7 +4746,8 @@ index 0000000..765be84 + pass + peak_rss = max(peak_rss, rss) + peak_swap = max(peak_swap, swap) -+ if args.rss_limit_gib is not None and rss > args.rss_limit_gib * (1024 ** 3): ++ # VmRSS 以 kB 為單位，門檻須換算成 kB（1 GiB = 1024 * 1024 kB） ++ if args.rss_limit_gib is not None and rss > args.rss_limit_gib * (1024 ** 2): + exceeded = True + process.send_signal(signal.SIGKILL) + break diff --git a/results/logs/common/mem_enforced_2g_arena128_summary.txt b/results/logs/common/mem_enforced_2g_arena128_summary.txt new file mode 100644 index 0000000000000000000000000000000000000000..a1a5c0e888b9df938ddae764f3cd499a2ace0c25 --- /dev/null +++ b/results/logs/common/mem_enforced_2g_arena128_summary.txt @@ -0,0 +1,9 @@ +v1 2 GiB arena128 run1 | Prompt: 6.7 t/s | Generation: 3.9 | peak "peak_gib": 1.887 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 2 GiB arena128 run1 | Prompt: 6.1 t/s | Generation: 3.8 | peak "peak_gib": 1.887 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2forced 2 GiB arena128 run1 | Prompt: 4.1 t/s | Generation: 3.9 | peak "peak_gib": 1.883 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v1 2 GiB arena128 run2 | Prompt: 6.1 t/s | Generation: 3.9 | peak "peak_gib": 1.887 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 2 GiB arena128 run2 | Prompt: 6.8 t/s | Generation: 3.9 | peak "peak_gib": 1.886 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2forced 2 GiB arena128 run2 | Prompt: 6.2 t/s | Generation: 3.8 | peak "peak_gib": 1.883 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v1 2 GiB arena128 run3 | Prompt: 6.3 t/s | Generation: 3.9 | peak "peak_gib": 1.887 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 2 GiB arena128 run3 | Prompt: 6.1 t/s | Generation: 3.8 | peak "peak_gib": 1.886 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2forced 2 GiB arena128 run3 | Prompt: 6.1 t/s | Generation: 3.9 | peak "peak_gib": 1.883 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 diff --git a/results/logs/common/mem_enforced_summary.txt b/results/logs/common/mem_enforced_summary.txt new file mode 100644 index 0000000000000000000000000000000000000000..66f279e985afbfb8c512ba47317ae5a70f02b9d5 --- /dev/null +++ b/results/logs/common/mem_enforced_summary.txt @@ -0,0 +1,60 @@ +native 2 GiB run1 | | peak "peak_gib": 3.039 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 2 GiB run2 | | peak "peak_gib": 2.422 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 2 GiB run3 | | peak "peak_gib": 2.214 | exceeded "limit_exceeded": true | exit "exit": -9 | +v1 2 GiB run1 | | peak "peak_gib": 2.001 | exceeded "limit_exceeded": true | exit "exit": -9 | +v1 2 GiB run2 | | peak "peak_gib": 2.0 | exceeded "limit_exceeded": true | exit "exit": -9 | +v1 2 GiB run3 | | peak "peak_gib": 2.001 | exceeded "limit_exceeded": true | exit "exit": -9 | +v2 2 GiB run1 | | peak "peak_gib": 2.0 | exceeded "limit_exceeded": true | exit "exit": -9 | +v2 2 GiB run2 | | peak "peak_gib": 2.001 | exceeded "limit_exceeded": true | exit "exit": -9 | +v2 2 GiB run3 | | peak "peak_gib": 2.0 | exceeded "limit_exceeded": true | exit "exit": -9 | +v2forced 2 GiB run1 | | peak "peak_gib": 2.001 | exceeded "limit_exceeded": true | exit "exit": -9 | +v2forced 2 GiB run2 | | peak "peak_gib": 2.001 | exceeded "limit_exceeded": true | exit "exit": -9 | +v2forced 2 GiB run3 | | peak "peak_gib": 2.001 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 4 GiB run1 | | peak "peak_gib": 4.195 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 4 GiB run2 | | peak "peak_gib": 4.123 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 4 GiB run3 | | peak "peak_gib": 4.274 | exceeded "limit_exceeded": true | exit "exit": -9 | +v1 4 GiB run1 | Prompt: 7.4 t/s | Generation: 4.7 | peak "peak_gib": 2.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 4 GiB run2 | Prompt: 5.9 t/s | Generation: 2.6 | peak "peak_gib": 2.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 4 GiB run3 | Prompt: 6.5 t/s | Generation: 4.7 | peak "peak_gib": 2.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 4 GiB run1 | Prompt: 7.3 t/s | Generation: 4.8 | peak "peak_gib": 2.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 4 GiB run2 | Prompt: 7.1 t/s | Generation: 4.8 | peak "peak_gib": 2.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 4 GiB run3 | Prompt: 7.0 t/s | Generation: 4.6 | peak "peak_gib": 2.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2forced 4 GiB run1 | Prompt: 7.2 t/s | Generation: 4.8 | peak "peak_gib": 2.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 4 GiB run2 | Prompt: 7.0 t/s | Generation: 4.7 | peak "peak_gib": 2.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 4 GiB run3 | Prompt: 7.0 t/s | Generation: 4.7 | peak "peak_gib": 2.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +native 8 GiB run1 | | peak "peak_gib": 8.143 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 8 GiB run2 | | peak "peak_gib": 8.08 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 8 GiB run3 | | peak "peak_gib": 8.179 | exceeded "limit_exceeded": true | exit "exit": -9 | +v1 8 GiB run1 | Prompt: 8.5 t/s | Generation: 4.8 | peak "peak_gib": 4.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 8 GiB run2 | Prompt: 9.7 t/s | Generation: 5.1 | peak "peak_gib": 4.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 8 GiB run3 | Prompt: 8.8 t/s | Generation: 4.8 | peak "peak_gib": 4.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 8 GiB run1 | Prompt: 9.3 t/s | Generation: 4.7 | peak "peak_gib": 4.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 8 GiB run2 | Prompt: 8.4 t/s | Generation: 4.9 | peak "peak_gib": 4.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 8 GiB run3 | Prompt: 9.7 t/s | Generation: 5.0 | peak "peak_gib": 4.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2forced 8 GiB run1 | Prompt: 8.5 t/s | Generation: 4.8 | peak "peak_gib": 4.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 8 GiB run2 | Prompt: 9.7 t/s | Generation: 5.3 | peak "peak_gib": 4.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 8 GiB run3 | Prompt: 9.1 t/s | Generation: 5.4 | peak "peak_gib": 4.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +native 16 GiB run1 | | peak "peak_gib": 16.544 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 16 GiB run2 | | peak "peak_gib": 16.025 | exceeded "limit_exceeded": true | exit "exit": -9 | +native 16 GiB run3 | | peak "peak_gib": 16.306 | exceeded "limit_exceeded": true | exit "exit": -9 | +v1 16 GiB run1 | Prompt: 11.2 t/s | Generation: 5.5 | peak "peak_gib": 11.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 16 GiB run2 | Prompt: 11.4 t/s | Generation: 5.6 | peak "peak_gib": 11.767 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 16 GiB run3 | Prompt: 10.9 t/s | Generation: 5.5 | peak "peak_gib": 11.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 16 GiB run1 | Prompt: 11.9 t/s | Generation: 5.4 | peak "peak_gib": 11.767 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 12 次，mode C 504 次 +v2 16 GiB run2 | Prompt: 12.1 t/s | Generation: 5.5 | peak "peak_gib": 11.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 12 次，mode C 504 次 +v2 16 GiB run3 | Prompt: 12.8 t/s | Generation: 5.6 | peak "peak_gib": 11.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 12 次，mode C 504 次 +v2forced 16 GiB run1 | Prompt: 12.6 t/s | Generation: 5.5 | peak "peak_gib": 11.76 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 16 GiB run2 | Prompt: 11.3 t/s | Generation: 5.4 | peak "peak_gib": 11.76 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 16 GiB run3 | Prompt: 13.1 t/s | Generation: 5.7 | peak "peak_gib": 11.76 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +native 32 GiB run1 | Prompt: 46.9 t/s | Generation: 21.8 | peak "peak_gib": 24.286 | exceeded "limit_exceeded": false | exit "exit": 0 | +native 32 GiB run2 | Prompt: 50.3 t/s | Generation: 21.6 | peak "peak_gib": 24.286 | exceeded "limit_exceeded": false | exit "exit": 0 | +native 32 GiB run3 | Prompt: 60.5 t/s | Generation: 28.8 | peak "peak_gib": 24.286 | exceeded "limit_exceeded": false | exit "exit": 0 | +v1 32 GiB run1 | Prompt: 11.8 t/s | Generation: 5.9 | peak "peak_gib": 19.766 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 32 GiB run2 | Prompt: 11.1 t/s | Generation: 5.7 | peak "peak_gib": 19.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v1 32 GiB run3 | Prompt: 11.3 t/s | Generation: 5.4 | peak "peak_gib": 19.765 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 0 次，mode C 516 次 +v2 32 GiB run1 | Prompt: 34.6 t/s | Generation: 16.5 | peak "peak_gib": 30.473 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2 32 GiB run2 | Prompt: 36.3 t/s | Generation: 14.6 | peak "peak_gib": 30.473 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2 32 GiB run3 | Prompt: 34.7 t/s | Generation: 13.8 | peak "peak_gib": 30.473 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 32 GiB run1 | Prompt: 13.3 t/s | Generation: 5.2 | peak "peak_gib": 19.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 32 GiB run2 | Prompt: 12.7 t/s | Generation: 5.4 | peak "peak_gib": 19.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 +v2forced 32 GiB run3 | Prompt: 12.6 t/s | Generation: 5.2 | peak "peak_gib": 19.759 | exceeded "limit_exceeded": false | exit "exit": 0 | 路徑計數：row-parallel 425 次，mode C 91 次 diff --git a/results/logs/native/mem_enforced_native_16g_run1.log b/results/logs/native/mem_enforced_native_16g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..b809b5dd7c672f4590a5dccffc9850f448314d88 --- /dev/null +++ b/results/logs/native/mem_enforced_native_16g_run1.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 17348012, "peak_gib": 16.544, "limit_gib": 16.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 1.72, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_16g_run2.log b/results/logs/native/mem_enforced_native_16g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..3fbcc61c3dae1823cd3047f0d5a86d3d7a9bdab7 --- /dev/null +++ b/results/logs/native/mem_enforced_native_16g_run2.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 16803060, "peak_gib": 16.025, "limit_gib": 16.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 1.55, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_16g_run3.log b/results/logs/native/mem_enforced_native_16g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..5c81da88ac54cc1e6b5691b72b31a66b3086a0d8 --- /dev/null +++ b/results/logs/native/mem_enforced_native_16g_run3.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 17097556, "peak_gib": 16.306, "limit_gib": 16.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 1.46, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_2g_run1.log b/results/logs/native/mem_enforced_native_2g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..ae342a4b2c0a284f5392aca6f3bf7cad19bd3aa0 --- /dev/null +++ b/results/logs/native/mem_enforced_native_2g_run1.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 3186608, "peak_gib": 3.039, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.52, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_2g_run2.log b/results/logs/native/mem_enforced_native_2g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..9e043a1bb111c604b72b6e6319cbd6b91493eba6 --- /dev/null +++ b/results/logs/native/mem_enforced_native_2g_run2.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 2539860, "peak_gib": 2.422, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.52, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_2g_run3.log b/results/logs/native/mem_enforced_native_2g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..ca4be2083dba4ce277d6adc3689d981243363dd7 --- /dev/null +++ b/results/logs/native/mem_enforced_native_2g_run3.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 2321728, "peak_gib": 2.214, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.53, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_32g_run1.log b/results/logs/native/mem_enforced_native_32g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..9e68ff57c303cf194f83dc03b8ef6d02e5cdc275 --- /dev/null +++ b/results/logs/native/mem_enforced_native_32g_run1.log @@ -0,0 +1,37 @@ + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one + +[ Prompt: 46.9 t/s | Generation: 21.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 25465768, "peak_gib": 24.286, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 3.97, "exit": 0, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_32g_run2.log b/results/logs/native/mem_enforced_native_32g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..72c5f59e4378a01ec0a8556d514b8ef34967835d --- /dev/null +++ b/results/logs/native/mem_enforced_native_32g_run2.log @@ -0,0 +1,37 @@ + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one + +[ Prompt: 50.3 t/s | Generation: 21.6 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 25465372, "peak_gib": 24.286, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 3.82, "exit": 0, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_32g_run3.log b/results/logs/native/mem_enforced_native_32g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..0327b900b7b812aa02f7398612c9c9b84d696fd4 --- /dev/null +++ b/results/logs/native/mem_enforced_native_32g_run3.log @@ -0,0 +1,37 @@ + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one + +[ Prompt: 60.5 t/s | Generation: 28.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 25465896, "peak_gib": 24.286, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 3.61, "exit": 0, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_4g_run1.log b/results/logs/native/mem_enforced_native_4g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..0ae0a3f0599ebbc41af50603310d065182b96954 --- /dev/null +++ b/results/logs/native/mem_enforced_native_4g_run1.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 4399196, "peak_gib": 4.195, "limit_gib": 4.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.65, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_4g_run2.log b/results/logs/native/mem_enforced_native_4g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..5fd6db56cbe526d41be3d48d5039da6954ca877b --- /dev/null +++ b/results/logs/native/mem_enforced_native_4g_run2.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 4323576, "peak_gib": 4.123, "limit_gib": 4.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.61, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_4g_run3.log b/results/logs/native/mem_enforced_native_4g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..37564528ae58e5fd9d7692cf7d7886f376b39dc2 --- /dev/null +++ b/results/logs/native/mem_enforced_native_4g_run3.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 4481508, "peak_gib": 4.274, "limit_gib": 4.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.66, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_8g_run1.log b/results/logs/native/mem_enforced_native_8g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..7e284a69f1ea91aafe193e5d618c873d096752e0 --- /dev/null +++ b/results/logs/native/mem_enforced_native_8g_run1.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 8538404, "peak_gib": 8.143, "limit_gib": 8.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 1.01, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_8g_run2.log b/results/logs/native/mem_enforced_native_8g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..792437216c3c5fd19e79ced7ac889c066ecc3adf --- /dev/null +++ b/results/logs/native/mem_enforced_native_8g_run2.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 8472136, "peak_gib": 8.08, "limit_gib": 8.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.95, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/native/mem_enforced_native_8g_run3.log b/results/logs/native/mem_enforced_native_8g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..fe61ca4f9fc7a72a7aa5fc7104e338f9c3b597c3 --- /dev/null +++ b/results/logs/native/mem_enforced_native_8g_run3.log @@ -0,0 +1 @@ +RUNMEM {"peak_kb": 8576276, "peak_gib": 8.179, "limit_gib": 8.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 0.98, "exit": -9, "cmd": ["env", "SDQ_PAGER=0", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_default_16g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_default_16g_run1.log similarity index 100% rename from results/logs/v2/mem_default_16g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_16g_run1.log diff --git a/results/logs/v2/mem_default_16g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_default_16g_run2.log similarity index 100% rename from results/logs/v2/mem_default_16g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_16g_run2.log diff --git a/results/logs/v2/mem_default_16g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_default_16g_run3.log similarity index 100% rename from results/logs/v2/mem_default_16g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_16g_run3.log diff --git a/results/logs/v2/mem_default_2g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_default_2g_run1.log similarity index 100% rename from results/logs/v2/mem_default_2g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_2g_run1.log diff --git a/results/logs/v2/mem_default_2g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_default_2g_run2.log similarity index 100% rename from results/logs/v2/mem_default_2g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_2g_run2.log diff --git a/results/logs/v2/mem_default_2g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_default_2g_run3.log similarity index 100% rename from results/logs/v2/mem_default_2g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_2g_run3.log diff --git a/results/logs/v2/mem_default_32g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_default_32g_run1.log similarity index 100% rename from results/logs/v2/mem_default_32g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_32g_run1.log diff --git a/results/logs/v2/mem_default_32g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_default_32g_run2.log similarity index 100% rename from results/logs/v2/mem_default_32g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_32g_run2.log diff --git a/results/logs/v2/mem_default_32g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_default_32g_run3.log similarity index 100% rename from results/logs/v2/mem_default_32g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_32g_run3.log diff --git a/results/logs/v2/mem_default_4g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_default_4g_run1.log similarity index 100% rename from results/logs/v2/mem_default_4g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_4g_run1.log diff --git a/results/logs/v2/mem_default_4g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_default_4g_run2.log similarity index 100% rename from results/logs/v2/mem_default_4g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_4g_run2.log diff --git a/results/logs/v2/mem_default_4g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_default_4g_run3.log similarity index 100% rename from results/logs/v2/mem_default_4g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_4g_run3.log diff --git a/results/logs/v2/mem_default_8g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_default_8g_run1.log similarity index 100% rename from results/logs/v2/mem_default_8g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_8g_run1.log diff --git a/results/logs/v2/mem_default_8g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_default_8g_run2.log similarity index 100% rename from results/logs/v2/mem_default_8g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_8g_run2.log diff --git a/results/logs/v2/mem_default_8g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_default_8g_run3.log similarity index 100% rename from results/logs/v2/mem_default_8g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_default_8g_run3.log diff --git a/results/logs/v2/mem_forced_16g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_16g_run1.log similarity index 100% rename from results/logs/v2/mem_forced_16g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_16g_run1.log diff --git a/results/logs/v2/mem_forced_16g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_16g_run2.log similarity index 100% rename from results/logs/v2/mem_forced_16g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_16g_run2.log diff --git a/results/logs/v2/mem_forced_16g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_16g_run3.log similarity index 100% rename from results/logs/v2/mem_forced_16g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_16g_run3.log diff --git a/results/logs/v2/mem_forced_2g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_2g_run1.log similarity index 100% rename from results/logs/v2/mem_forced_2g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_2g_run1.log diff --git a/results/logs/v2/mem_forced_2g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_2g_run2.log similarity index 100% rename from results/logs/v2/mem_forced_2g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_2g_run2.log diff --git a/results/logs/v2/mem_forced_2g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_2g_run3.log similarity index 100% rename from results/logs/v2/mem_forced_2g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_2g_run3.log diff --git a/results/logs/v2/mem_forced_32g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_32g_run1.log similarity index 100% rename from results/logs/v2/mem_forced_32g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_32g_run1.log diff --git a/results/logs/v2/mem_forced_32g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_32g_run2.log similarity index 100% rename from results/logs/v2/mem_forced_32g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_32g_run2.log diff --git a/results/logs/v2/mem_forced_32g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_32g_run3.log similarity index 100% rename from results/logs/v2/mem_forced_32g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_32g_run3.log diff --git a/results/logs/v2/mem_forced_4g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_4g_run1.log similarity index 100% rename from results/logs/v2/mem_forced_4g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_4g_run1.log diff --git a/results/logs/v2/mem_forced_4g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_4g_run2.log similarity index 100% rename from results/logs/v2/mem_forced_4g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_4g_run2.log diff --git a/results/logs/v2/mem_forced_4g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_4g_run3.log similarity index 100% rename from results/logs/v2/mem_forced_4g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_4g_run3.log diff --git a/results/logs/v2/mem_forced_8g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_8g_run1.log similarity index 100% rename from results/logs/v2/mem_forced_8g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_8g_run1.log diff --git a/results/logs/v2/mem_forced_8g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_8g_run2.log similarity index 100% rename from results/logs/v2/mem_forced_8g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_8g_run2.log diff --git a/results/logs/v2/mem_forced_8g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_forced_8g_run3.log similarity index 100% rename from results/logs/v2/mem_forced_8g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_forced_8g_run3.log diff --git a/results/logs/common/mem_summary.txt b/results/logs/superseded_runmem_threshold_bug/mem_summary.txt similarity index 100% rename from results/logs/common/mem_summary.txt rename to results/logs/superseded_runmem_threshold_bug/mem_summary.txt diff --git a/results/logs/v1/mem_v1_16g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_16g_run1.log similarity index 100% rename from results/logs/v1/mem_v1_16g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_16g_run1.log diff --git a/results/logs/v1/mem_v1_16g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_16g_run2.log similarity index 100% rename from results/logs/v1/mem_v1_16g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_16g_run2.log diff --git a/results/logs/v1/mem_v1_16g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_16g_run3.log similarity index 100% rename from results/logs/v1/mem_v1_16g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_16g_run3.log diff --git a/results/logs/v1/mem_v1_2g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_2g_run1.log similarity index 100% rename from results/logs/v1/mem_v1_2g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_2g_run1.log diff --git a/results/logs/v1/mem_v1_2g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_2g_run2.log similarity index 100% rename from results/logs/v1/mem_v1_2g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_2g_run2.log diff --git a/results/logs/v1/mem_v1_2g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_2g_run3.log similarity index 100% rename from results/logs/v1/mem_v1_2g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_2g_run3.log diff --git a/results/logs/v1/mem_v1_32g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_32g_run1.log similarity index 100% rename from results/logs/v1/mem_v1_32g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_32g_run1.log diff --git a/results/logs/v1/mem_v1_32g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_32g_run2.log similarity index 100% rename from results/logs/v1/mem_v1_32g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_32g_run2.log diff --git a/results/logs/v1/mem_v1_32g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_32g_run3.log similarity index 100% rename from results/logs/v1/mem_v1_32g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_32g_run3.log diff --git a/results/logs/v1/mem_v1_4g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_4g_run1.log similarity index 100% rename from results/logs/v1/mem_v1_4g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_4g_run1.log diff --git a/results/logs/v1/mem_v1_4g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_4g_run2.log similarity index 100% rename from results/logs/v1/mem_v1_4g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_4g_run2.log diff --git a/results/logs/v1/mem_v1_4g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_4g_run3.log similarity index 100% rename from results/logs/v1/mem_v1_4g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_4g_run3.log diff --git a/results/logs/v1/mem_v1_8g_run1.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_8g_run1.log similarity index 100% rename from results/logs/v1/mem_v1_8g_run1.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_8g_run1.log diff --git a/results/logs/v1/mem_v1_8g_run2.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_8g_run2.log similarity index 100% rename from results/logs/v1/mem_v1_8g_run2.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_8g_run2.log diff --git a/results/logs/v1/mem_v1_8g_run3.log b/results/logs/superseded_runmem_threshold_bug/mem_v1_8g_run3.log similarity index 100% rename from results/logs/v1/mem_v1_8g_run3.log rename to results/logs/superseded_runmem_threshold_bug/mem_v1_8g_run3.log diff --git a/results/logs/v2/v2_32g_run1.log b/results/logs/superseded_runmem_threshold_bug/v2_32g_run1.log similarity index 100% rename from results/logs/v2/v2_32g_run1.log rename to results/logs/superseded_runmem_threshold_bug/v2_32g_run1.log diff --git a/results/logs/v2/v2_32g_run2.log b/results/logs/superseded_runmem_threshold_bug/v2_32g_run2.log similarity index 100% rename from results/logs/v2/v2_32g_run2.log rename to results/logs/superseded_runmem_threshold_bug/v2_32g_run2.log diff --git a/results/logs/v2/v2_32g_run3.log b/results/logs/superseded_runmem_threshold_bug/v2_32g_run3.log similarity index 100% rename from results/logs/v2/v2_32g_run3.log rename to results/logs/superseded_runmem_threshold_bug/v2_32g_run3.log diff --git a/results/logs/common/v2_mem_summary.txt b/results/logs/superseded_runmem_threshold_bug/v2_mem_summary.txt similarity index 100% rename from results/logs/common/v2_mem_summary.txt rename to results/logs/superseded_runmem_threshold_bug/v2_mem_summary.txt diff --git a/results/logs/v1/mem_enforced_v1_16g_run1.log b/results/logs/v1/mem_enforced_v1_16g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..7e0a3ff88041083b69fed9ac159fe03c3bf34c4d --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_16g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.2 t/s | Generation: 5.5 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12338060, "peak_gib": 11.766, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.88, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_16g_run2.log b/results/logs/v1/mem_enforced_v1_16g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..fca86da8e652288a74f096f352d0e01f761048b6 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_16g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.4 t/s | Generation: 5.6 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12338312, "peak_gib": 11.767, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.75, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_16g_run3.log b/results/logs/v1/mem_enforced_v1_16g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..fe82ba3c14fd38ec1322a38471569075e780f7d4 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_16g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 10.9 t/s | Generation: 5.5 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12337976, "peak_gib": 11.766, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.79, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_2g_arena128_run1.log b/results/logs/v1/mem_enforced_v1_2g_arena128_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..7723b11b60c7d16a500adbabb33aa870cec98319 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_2g_arena128_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.7 t/s | Generation: 3.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1978556, "peak_gib": 1.887, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 4.93, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_2g_arena128_run2.log b/results/logs/v1/mem_enforced_v1_2g_arena128_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..f612c7102f3ebbc0bd5d36ff710622c108f810eb --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_2g_arena128_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.1 t/s | Generation: 3.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1978224, "peak_gib": 1.887, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.11, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_2g_arena128_run3.log b/results/logs/v1/mem_enforced_v1_2g_arena128_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..be33b08ac1683f72c8ac68e3361f29a54143b2bc --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_2g_arena128_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.3 t/s | Generation: 3.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1978428, "peak_gib": 1.887, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.05, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_2g_run1.log b/results/logs/v1/mem_enforced_v1_2g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..a15fc86ac432a83361666b774763fb73dac00cf3 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_2g_run1.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2098108, "peak_gib": 2.001, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 1.98, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_2g_run2.log b/results/logs/v1/mem_enforced_v1_2g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..eff72d3063a9a8a9add7d0f350ee7b850b47c237 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_2g_run2.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2097452, "peak_gib": 2.0, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 1.96, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_2g_run3.log b/results/logs/v1/mem_enforced_v1_2g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..e884ca31f5fc4ab030654bc8f772b3246a4765e3 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_2g_run3.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2097776, "peak_gib": 2.001, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.08, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_32g_run1.log b/results/logs/v1/mem_enforced_v1_32g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..2e228d03c25ece820468942b379dc291206124b2 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_32g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.8 t/s | Generation: 5.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 20725644, "peak_gib": 19.766, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 14.45, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_32g_run2.log b/results/logs/v1/mem_enforced_v1_32g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..e1ff0c1cf5d973b238d30f43b621b31d84cfc154 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_32g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.1 t/s | Generation: 5.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 20725592, "peak_gib": 19.765, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 14.54, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_32g_run3.log b/results/logs/v1/mem_enforced_v1_32g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..765c20c37ddd7ff9e7967fc00156929eeb6b83c7 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_32g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.3 t/s | Generation: 5.4 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 20725400, "peak_gib": 19.765, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 14.87, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_4g_run1.log b/results/logs/v1/mem_enforced_v1_4g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..ee3db4fab40fa2efb186d0d9e1e232cc1e4ae7ac --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_4g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.4 t/s | Generation: 4.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2899280, "peak_gib": 2.765, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.14, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_4g_run2.log b/results/logs/v1/mem_enforced_v1_4g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..6098317af48f8fd3e09bec61aaf2cefaba51f7c1 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_4g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 5.9 t/s | Generation: 2.6 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2899676, "peak_gib": 2.765, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 6.78, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_4g_run3.log b/results/logs/v1/mem_enforced_v1_4g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..f49b96482b5a3c88812d7ee265ca69d4082229bb --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_4g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.5 t/s | Generation: 4.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2899628, "peak_gib": 2.765, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.36, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_8g_run1.log b/results/logs/v1/mem_enforced_v1_8g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..5d730e7a6f23712704fd2ddf90301251937a7789 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_8g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 8.5 t/s | Generation: 4.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4997120, "peak_gib": 4.766, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.98, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_8g_run2.log b/results/logs/v1/mem_enforced_v1_8g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..c5a94cc68938650e71ae5d57a46df1bbebd61f23 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_8g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 9.7 t/s | Generation: 5.1 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4997216, "peak_gib": 4.766, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.92, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v1/mem_enforced_v1_8g_run3.log b/results/logs/v1/mem_enforced_v1_8g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..1eb9248fc9c9a7e8b09c1d50ece86116dc0e6938 --- /dev/null +++ b/results/logs/v1/mem_enforced_v1_8g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 8.8 t/s | Generation: 4.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4996712, "peak_gib": 4.765, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.94, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=0", "SDQ_ROW_PARALLEL=0", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_16g_run1.log b/results/logs/v2/mem_enforced_v2_16g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..86d53ca8f3fb6fe6ffacb39f677dcfbb60146a74 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_16g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 12 次，mode C 504 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.9 t/s | Generation: 5.4 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12338184, "peak_gib": 11.767, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.84, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_16g_run2.log b/results/logs/v2/mem_enforced_v2_16g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..862206d633ae28908576973897e051b9bf631ad6 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_16g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 12 次，mode C 504 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 12.1 t/s | Generation: 5.5 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12337856, "peak_gib": 11.766, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 10.17, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_16g_run3.log b/results/logs/v2/mem_enforced_v2_16g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..33164237cf7333c92b253f56ddef7047462517c2 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_16g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 12 次，mode C 504 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 12.8 t/s | Generation: 5.6 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12337720, "peak_gib": 11.766, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.92, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_2g_arena128_run1.log b/results/logs/v2/mem_enforced_v2_2g_arena128_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..92f7dfacc6a1bd38faba95afc6b66a09fc2925b9 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_2g_arena128_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.1 t/s | Generation: 3.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1978800, "peak_gib": 1.887, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.13, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_2g_arena128_run2.log b/results/logs/v2/mem_enforced_v2_2g_arena128_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..7d8989d7369a88b5035cd8921c0260ab49689a60 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_2g_arena128_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.8 t/s | Generation: 3.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1978024, "peak_gib": 1.886, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.1, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_2g_arena128_run3.log b/results/logs/v2/mem_enforced_v2_2g_arena128_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..882bf9c8cd04e19438c19166055f28856d5fe692 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_2g_arena128_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.1 t/s | Generation: 3.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1978100, "peak_gib": 1.886, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.13, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_2g_run1.log b/results/logs/v2/mem_enforced_v2_2g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..1b521b31bc72da0a996a2bd34aedc9b5b2e8c076 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_2g_run1.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2097368, "peak_gib": 2.0, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.05, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_2g_run2.log b/results/logs/v2/mem_enforced_v2_2g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..5c6868d56ad591753c58deea88c57fa6803547bd --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_2g_run2.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2097824, "peak_gib": 2.001, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.16, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_2g_run3.log b/results/logs/v2/mem_enforced_v2_2g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..3124892a6950bce20012c4c052cfd7428465519c --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_2g_run3.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2097304, "peak_gib": 2.0, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.01, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_32g_run1.log b/results/logs/v2/mem_enforced_v2_32g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..80d20d7940533800f03dd0d59fcdfa7bf082ed3b --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_32g_run1.log @@ -0,0 +1,58 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=12061 arena=28.71 GiB (RAM 實測 RSS=28.77 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 12061 +[sdq] prefetch=off fadvise_dontneed=1 +[sdq] 已預載全部 expert（12061 slots）→ 之後 row-parallel 可用 +[sdq] 已預載全部 expert（12061 slots）→ 之後 row-parallel 可用 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=12061 arena=28.71 GiB (RAM 實測 RSS=28.88 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 12061 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 34.6 t/s | Generation: 16.5 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 31953332, "peak_gib": 30.473, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 31.83, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=29400", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_32g_run2.log b/results/logs/v2/mem_enforced_v2_32g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..820dbb5f03e37882766e6df1ce030338964e2e5f --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_32g_run2.log @@ -0,0 +1,58 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=12061 arena=28.71 GiB (RAM 實測 RSS=28.77 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 12061 +[sdq] prefetch=off fadvise_dontneed=1 +[sdq] 已預載全部 expert（12061 slots）→ 之後 row-parallel 可用 +[sdq] 已預載全部 expert（12061 slots）→ 之後 row-parallel 可用 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=12061 arena=28.71 GiB (RAM 實測 RSS=28.88 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 12061 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 36.3 t/s | Generation: 14.6 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 31952820, "peak_gib": 30.473, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 29.92, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=29400", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_32g_run3.log b/results/logs/v2/mem_enforced_v2_32g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..c0ac0a389e17cf13440a6ed0ee4f336a3bf7f3ae --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_32g_run3.log @@ -0,0 +1,58 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=12061 arena=28.71 GiB (RAM 實測 RSS=28.77 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 12061 +[sdq] prefetch=off fadvise_dontneed=1 +[sdq] 已預載全部 expert（12061 slots）→ 之後 row-parallel 可用 +[sdq] 已預載全部 expert（12061 slots）→ 之後 row-parallel 可用 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=12061 arena=28.71 GiB (RAM 實測 RSS=28.88 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 12061 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 34.7 t/s | Generation: 13.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 31953088, "peak_gib": 30.473, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 29.97, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=29400", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_4g_run1.log b/results/logs/v2/mem_enforced_v2_4g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..9713ee6060a571f7fcf8314aaac77e7c0a7afd33 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_4g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.3 t/s | Generation: 4.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2899500, "peak_gib": 2.765, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.1, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_4g_run2.log b/results/logs/v2/mem_enforced_v2_4g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..7efc9634c88531dbb83ae6882883366094186bf7 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_4g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.1 t/s | Generation: 4.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2899296, "peak_gib": 2.765, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.15, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_4g_run3.log b/results/logs/v2/mem_enforced_v2_4g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..7b67f98e5a385130cd2b201e885cf1afe085e268 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_4g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.0 t/s | Generation: 4.6 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2899496, "peak_gib": 2.765, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.26, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_8g_run1.log b/results/logs/v2/mem_enforced_v2_8g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..03a8c0c19097f7f3c3bd7d5321d7303b1b507af5 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_8g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 9.3 t/s | Generation: 4.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4996484, "peak_gib": 4.765, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 6.22, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_8g_run2.log b/results/logs/v2/mem_enforced_v2_8g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..b2a9b45c4d717b68e2037cc2a1846f13ddf41ffd --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_8g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 8.4 t/s | Generation: 4.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4996568, "peak_gib": 4.765, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 6.0, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2_8g_run3.log b/results/logs/v2/mem_enforced_v2_8g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..29e788ae4a0985d7cd5943821011e711c450908f --- /dev/null +++ b/results/logs/v2/mem_enforced_v2_8g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 0 次，mode C 516 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.17 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 9.7 t/s | Generation: 5.0 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4996216, "peak_gib": 4.765, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.97, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_16g_run1.log b/results/logs/v2/mem_enforced_v2forced_16g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..9af48fff03420d3f8c88e1b8e69fc738b1a8806e --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_16g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 12.6 t/s | Generation: 5.5 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12331172, "peak_gib": 11.76, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.82, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_16g_run2.log b/results/logs/v2/mem_enforced_v2forced_16g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..dd69e04f673bebbf687c9768131fd24e443be274 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_16g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 11.3 t/s | Generation: 5.4 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12331580, "peak_gib": 11.76, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 9.96, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_16g_run3.log b/results/logs/v2/mem_enforced_v2forced_16g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..6d4d0e106cd0381b8a599db7151e786e6a96d9b6 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_16g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=4201 arena=10.00 GiB (RAM 實測 RSS=10.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 4201 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 13.1 t/s | Generation: 5.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 12331460, "peak_gib": 11.76, "limit_gib": 16.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 10.06, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=10240", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_2g_arena128_run1.log b/results/logs/v2/mem_enforced_v2forced_2g_arena128_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..4d3658b0970a5e625aee3fc96e0d1108175a73bd --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_2g_arena128_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 4.1 t/s | Generation: 3.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1974384, "peak_gib": 1.883, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 7.25, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_2g_arena128_run2.log b/results/logs/v2/mem_enforced_v2forced_2g_arena128_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..c2cac9b11aff6244d528684869d00bd324d6763f --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_2g_arena128_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.2 t/s | Generation: 3.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1974788, "peak_gib": 1.883, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.13, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_2g_arena128_run3.log b/results/logs/v2/mem_enforced_v2forced_2g_arena128_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..5bf102588db9241fe19a46409a1260c077d3a9f8 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_2g_arena128_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.18 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=52 arena=0.12 GiB (RAM 實測 RSS=0.29 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 52 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 6.1 t/s | Generation: 3.9 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 1974248, "peak_gib": 1.883, "limit_gib": 2.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.12, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=128", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_2g_run1.log b/results/logs/v2/mem_enforced_v2forced_2g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..a2bb39acd29b787a51f0a25d21475fe32d27b9f2 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_2g_run1.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2098020, "peak_gib": 2.001, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.26, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_2g_run2.log b/results/logs/v2/mem_enforced_v2forced_2g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..e2afea9e425434208c46dd0566f02b881f2e719e --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_2g_run2.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2098024, "peak_gib": 2.001, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.27, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_2g_run3.log b/results/logs/v2/mem_enforced_v2forced_2g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..8cd7b9ddfbdfd0f2530f890decbb6c10712e61c9 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_2g_run3.log @@ -0,0 +1,12 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=105 arena=0.25 GiB (RAM 實測 RSS=0.31 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 105 +[sdq] prefetch=off fadvise_dontneed=1 +RUNMEM {"peak_kb": 2098044, "peak_gib": 2.001, "limit_gib": 2.0, "limit_exceeded": true, "swap_kb": 0, "wall_s": 2.27, "exit": -9, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=256", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_32g_run1.log b/results/logs/v2/mem_enforced_v2forced_32g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..fd90a1497a08990cd9bae0f7078f411e6c64407d --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_32g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 13.3 t/s | Generation: 5.2 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 20718492, "peak_gib": 19.759, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 15.86, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_32g_run2.log b/results/logs/v2/mem_enforced_v2forced_32g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..447f6727754d93fc7c550e7c91d1e7002fb4f3de --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_32g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 12.7 t/s | Generation: 5.4 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 20718524, "peak_gib": 19.759, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 16.32, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_32g_run3.log b/results/logs/v2/mem_enforced_v2forced_32g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..3325d2d2e1288e754153e7a4974934a3829b5233 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_32g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=7561 arena=18.00 GiB (RAM 實測 RSS=18.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 7561 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 12.6 t/s | Generation: 5.2 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 20719172, "peak_gib": 19.759, "limit_gib": 32.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 17.05, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=18432", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_4g_run1.log b/results/logs/v2/mem_enforced_v2forced_4g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..dd3ed5178f88047cbae8a8e9c1541b97b301bfca --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_4g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.2 t/s | Generation: 4.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2893488, "peak_gib": 2.759, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.2, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_4g_run2.log b/results/logs/v2/mem_enforced_v2forced_4g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..5e15fb54d853478a59bad4cee9b7bc0b67694537 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_4g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.0 t/s | Generation: 4.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2893152, "peak_gib": 2.759, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.24, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_4g_run3.log b/results/logs/v2/mem_enforced_v2forced_4g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..85263f52bacd37f4f7fd625d773345343a727664 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_4g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=420 arena=1.00 GiB (RAM 實測 RSS=1.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 420 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 7.0 t/s | Generation: 4.7 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 2892900, "peak_gib": 2.759, "limit_gib": 4.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.2, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=1024", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_8g_run1.log b/results/logs/v2/mem_enforced_v2forced_8g_run1.log new file mode 100644 index 0000000000000000000000000000000000000000..0f9111e7deb9bff0722c85b82753ec3be92d8829 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_8g_run1.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 8.5 t/s | Generation: 4.8 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4989764, "peak_gib": 4.759, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.96, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_8g_run2.log b/results/logs/v2/mem_enforced_v2forced_8g_run2.log new file mode 100644 index 0000000000000000000000000000000000000000..c15fd405acc26da27d3e720066b39c1121a839b1 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_8g_run2.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 9.7 t/s | Generation: 5.3 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4990000, "peak_gib": 4.759, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 5.87, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/logs/v2/mem_enforced_v2forced_8g_run3.log b/results/logs/v2/mem_enforced_v2forced_8g_run3.log new file mode 100644 index 0000000000000000000000000000000000000000..f44ea3687e98a3e6ccef5a8f9ee0dd9e6f0ddff8 --- /dev/null +++ b/results/logs/v2/mem_enforced_v2forced_8g_run3.log @@ -0,0 +1,56 @@ +[sdq] MoE 層 47 / 共 49 層（dense lead 2）：expert 權重合計 21.47 GiB、12032 個 expert +[sdq] 統一槽位 2555904 B（最大 expert 大小；平均 1915708 B） +[sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） +[sdq] I/O 段平行：24 條讀取執行緒 × 1096 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.06 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + + +Loading model... + +▄▄ ▄▄ +██ ██ +██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ +██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ +██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ + ██ ██ + ▀▀ ▀▀ + +build : b1-970b51c +model : /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +ftype : Q4_K - Medium +modalities : text + +available commands: + /exit or Ctrl+C stop or exit + /regen regenerate the last response + /clear clear the chat history + /read add a text file + /glob add text files using globbing pattern + + + +> Explain in one sentence why the sky is blue. + +[Start thinking] + +The user is asking for a one +[sdq] MoE op 路徑計數：row-parallel 425 次，mode C 91 次 +[sdq] page-map: n_layer=49 n_expert=256 n_expert_used=6 n_embd=2048 n_ff_exp=512 | L2E0 gate=986105536/450560 up=1104733888/450560 down=799327936/720896 (off/size) ne0=2048 ne1=512 d_ne0=512 d_ne1=2048 +[sdq] expert pager ready: /tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf +[sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） +[sdq] slot=2.44 MiB (2555904 B) slots=1260 arena=3.00 GiB (RAM 實測 RSS=3.16 GiB) +[sdq] evict_policy=lru stale_window=640 +[sdq] 槽位：單一尺寸 2.44 MiB × 1260 +[sdq] prefetch=off fadvise_dontneed=1 + +[ Prompt: 9.1 t/s | Generation: 5.4 t/s ] + + +Exiting... +RUNMEM {"peak_kb": 4990132, "peak_gib": 4.759, "limit_gib": 8.0, "limit_exceeded": false, "swap_kb": 0, "wall_s": 6.09, "exit": 0, "cmd": ["env", "SDQ_PAGER=1", "SDQ_PREFETCH=0", "SDQ_ARENA_MB=3072", "SDQ_PRELOAD=1", "SDQ_ROW_PARALLEL=1", "/tmp/opencode/flint-work/llama.cpp/build/bin/llama-cli", "-m", "/tmp/opencode/flint-work/llama.cpp/models/flint-UD-Q4_K_XL.gguf", "-c", "128", "-b", "32", "-ub", "16", "-p", "Explain in one sentence why the sky is blue.", "-n", "8", "-t", "16", "--temp", "0", "-st", "--simple-io"], "method": "procfs-rss-watchdog"} diff --git a/results/test-status.md b/results/test-status.md index 7db66965d087b735c2e4c1307ea68e30f80fa18d..63e9a7d89b69e8444e63526aac9c57d9dcc74f59 100644 --- a/results/test-status.md +++ b/results/test-status.md @@ -2,9 +2,9 @@ Environment: Intel Xeon Platinum 8559C, container quota 16 CPUs, cgroup v2 `memory.max` 104,000,000,000 bytes (~96.9 GiB), `memory.swap.max` 2,936,848,384 bytes (~2.74 GiB), Amazon Linux 2023, kernel 6.18.48, CPU only. The container's cgroup v2 mount is read-only and there is no systemd manager, so no cgroup-enforced memory limit could be applied. -llama.cpp base: `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` plus `flint-epq.patch` (sha256 `f6ae3fd33dcd29a132e37bfebacc6c452778e752266bb963718f8ec6f9522572`). +llama.cpp base: `970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc` plus `flint-epq.patch` (sha256 `b8a4e77af3743bd35bd4c14258c70c12031e60ba264f95ab93593519ea0ab08c`). -Versions: **v1** = expert paging, mode C (`SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`). **v2** = v1 + preload + row-parallel for gated experts (`SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1`, arena ≥ 28.6 GiB). **native** = `SDQ_PAGER=0`. +Versions: **native** = `SDQ_PAGER=0`. **v1** = expert paging, mode C (`SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`). **v2** = v1 + preload + row-parallel for gated experts (`SDQ_PRELOAD=1 SDQ_ROW_PARALLEL=1`, arena ≥ 28.6 GiB). **v2 forced** = v2 with `SDQ_ROWPAR_MAX_MISS=100000` (ablation). ## Checks @@ -13,7 +13,7 @@ Versions: **v1** = expert paging, mode C (`SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`). * | Clean checkout of `970b51c` + patch applies; CPU build | v1, v2 | PASS (fresh worktree, configure command from README) | | Flint GGUF header / expert map | — | PASS: 49 layers (2 dense + 47 MoE), 256 experts, top-6, 12,032 routed experts, 21.47 GiB routed expert weights | | Expert type coverage (`IQ3_S`, `IQ4_XS`, `Q4_K`, `Q5_K`, `Q6_K`, `Q8_0`) | — | PASS: all types have dot-product kernels; gate and up share one activation type in all 47 layers | -| Smoke test (README command, `-c 2048`, arena 512 MiB, `--single-turn`) | v1 | PASS: exit 0, answer produced, peak process-tree RSS 2.392 GiB, swap 0 | +| Smoke test (README command, `-c 2048`, arena 512 MiB, `--single-turn`) | v1 | PASS: exit 0, answer produced, peak process-tree RSS 2.394 GiB, swap 0 | | Greedy, 24 tokens, 8 threads | v1, v2 | PASS: identical to native | | Greedy, 24 tokens, 16 threads | v1, v2 | v1 = v2; both differ from native at char 13 (near-tie argmax flip) | | Prefill perplexity, `-ub 64`, 8 threads, 4 chunks | native / v1 / v2 | 10.7790 / 10.8014 / 10.8014 | @@ -21,40 +21,44 @@ Versions: **v1** = expert paging, mode C (`SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`). * | Decode KL vs native, `-ub 1`, 16 threads, 2 chunks | v1 | mean 0.044057; max 3.16; top-1 92.52 % | | Decode KL vs native, `-ub 1`, 16 threads, 2 chunks | v2 | mean 0.044057; max 3.16; top-1 92.52 % | | Decode v2 vs v1, `-ub 1`, 16 threads | v2 vs v1 | mean KLD 1e-6 (max 3.8e-5); top-1 100 %. Row-parallel is numerically equivalent to mode C but not bit-identical | +| Decode, v2 forced with 1 GiB arena, `-ub 1`, KL vs native | v2 forced | mean 0.044057; top-1 92.52 %; identical to v1 at reported precision | +| Decode, mode C with 1 GiB arena, `-ub 1`, KL vs native | v1 | mean 0.044057; identical to full-arena v1 (eviction does not change results) | | Control: native `-ub 64` vs native `-ub 1`, 16 threads | native | mean KLD 0.034464 ± 0.008439; top-1 90.16 % | | Control: native `-ub 16` vs `-ub 64`, prefill | native | mean KLD 0.024005 ± 0.003389; top-1 93.11 % | | Layer-wise MoE output vs native, layers 2–6 (same input) | v1 | max relative difference ≤ 1.6e-5 | -| Row-parallel active check (`SDQ_DBG_OP`) | v2 | `rowpar=1` on decode ops with 6 needed experts at 16 threads; `rowpar=0` with `SDQ_ROW_PARALLEL=0` | +| Row-parallel active check | v2 | `rowpar=1` on decode ops at 16 threads; `rowpar=0` with `SDQ_ROW_PARALLEL=0` | +| Path counter (`MoE op 路徑計數`) | v2 | 425 row-parallel / 91 mode C at 32 GiB; 0 / 516 at 2–8 GiB; 12 / 504 at 16 GiB | | Preload check | v2 | "已預載全部 expert（12061 slots）" with a 28.7 GiB arena | -| Throughput, context 512, batch 128, ubatch 32, 32 tokens, 16 threads (3 runs, median) | native / v1 / v2 | Generation 21.5 / 5.6 / 16.2 t/s; prompt 54.4 / 13.0 / 32.4 t/s. Peak RSS 24.3 / 19.8 / 30.5 GiB | -| Memory tiers 2 / 4 / 8 / 16 / 32 GiB, 16 threads (3 runs each) | v1 | PASS: all 15 runs exit 0, swap 0, never over threshold; peaks 2.014 / 2.765 / 4.766 / 11.767 / 19.766 GiB | -| Memory tiers 2 / 4 / 8 / 16 / 32 GiB, v2 as designed, 16 threads (3 runs each) | v2 | PASS: all 15 runs exit 0, swap 0. Peaks 2.014 / 2.765 / 4.766 / 11.767 / 30.473 GiB. RP / mode C ops: 0/516, 0/516, 0/516, 12/504, 425/91 | -| Memory tiers 2 / 4 / 8 / 16 / 32 GiB, v2 forced row-parallel (`SDQ_ROWPAR_MAX_MISS=100000`), 16 threads (3 runs each) | v2 forced | PASS: all 15 runs exit 0, swap 0. Peaks 2.009 / 2.759 / 4.759 / 11.760 / 19.759 GiB; 425 RP ops at every tier | -| Forced row-parallel with 1 GiB arena, decode `-ub 1`, KL vs native | v2 forced | mean 0.044057; top-1 92.52 %; identical to v1 full-arena at reported precision (0.044057) | -| Mode C with 1 GiB arena, decode `-ub 1`, KL vs native | v1 | mean 0.044057; identical to full-arena v1 (eviction does not change results) | -| v2 path counter at 32 GiB (full arena) | v2 | row-parallel 425, mode C 91 (prefill) | -| Fresh checkout of new patch (sha `f6ae3fd…`): build and path counter | v2 | PASS: 425 RP / 91 mode C reproduced | -| 32 GiB tier, 28.7 GiB arena, 16 threads (3 runs) | v2 | PASS: peak 30.473 GiB, swap 0, never over threshold | +| Fresh checkout of patch (sha `b8a4e77…`): build, counter reproduced | v2 | PASS: 425 / 91 reproduced (run before the watchdog fix; patch re-verified to apply cleanly) | +| Throughput, context 512, batch 128, ubatch 32, 32 tokens, 16 threads (3 runs, median) | native / v1 / v2 | Generation 21.5 / 5.6 / 16.2 t/s; prompt 54.4 / 13.0 / 32.4 t/s. Peak RSS 24.3 / 19.8 / 30.5 GiB. No watchdog applied | +| Memory tiers, **enforced**, native, 2 / 4 / 8 / 16 / 32 GiB, 16 threads (3 runs each) | native | 2, 4, 8, 16 GiB: killed in 3 / 3 runs each (peaks 2.2–3.0, 4.1–4.3, 8.1–8.2, 16.0–16.5 GiB at kill). 32 GiB: pass, peak 24.286 GiB, generation 21.8 t/s (median) | +| Memory tiers, **enforced**, v1, 2 / 4 / 8 / 16 / 32 GiB, 16 threads (3 runs each) | v1 | 2 GiB with 256 MiB arena: killed (peak 2.000–2.001 GiB). 4 / 8 / 16 / 32 GiB: pass, peaks 2.765 / 4.766 / 11.766 / 19.765 GiB | +| Memory tier 2 GiB with 128 MiB arena (3 runs each) | v1, v2, v2 forced | PASS: peaks 1.887 / 1.886 / 1.883 GiB | +| Memory tiers, **enforced**, v2 as designed, 2 / 4 / 8 / 16 / 32 GiB | v2 | 2 GiB (256 MiB): killed. 4 / 8 / 16 GiB: pass, peaks 2.765 / 4.765 / 11.766 GiB, 0 / 0 / 12 RP ops. 32 GiB (28.7 GiB arena): pass, peak 30.473 GiB, 425 RP ops | +| Memory tiers, **enforced**, v2 forced, 2 / 4 / 8 / 16 / 32 GiB | v2 forced | 2 GiB (256 MiB): killed. 4 / 8 / 16 / 32 GiB: pass, peaks 2.759 / 4.759 / 11.760 / 19.759 GiB, 425 RP ops each | | cgroup-v2 `MemoryMax` runs (`tools/runmem_cgroup.py`) | — | NOT RUN: no writable cgroup v2 / systemd here. The script is untested | ## Defects found and fixed -1. **Bad debug hook (debug path only):** `sdq_maybe_dump_tensor` (enabled by `SDQ_MOE_OUT_DUMP`) wrote the tensor to a file but never copied it into its output, so downstream nodes read uninitialized memory. Fixed; the production path never used this node. -2. **SwiGLU clamp mismatch:** the pager clamped `silu(gate)` to `[-limit, limit]`; upstream clamps only the upper bound. Fixed to match upstream. -3. **`swiglu_limit` was dropped (found in this pass):** `sdq_build_moe` ignored its `swiglu_limit` argument and the op always used 0. Fixed; the value now reaches the op. This checkpoint has limit 0, so the numbers are unaffected. -4. **Published patch was stale:** the patch on Hugging Face predated these fixes. Re-generated from the verified tree and re-uploaded. -5. **Smoke command in the model card was not the one that was run.** The card now shows the executed command. -6. **`tools/runmem_cgroup.py` and the cgroup claim were presented as usable.** Now labeled untested. -7. **Earlier memory-tier throughput came from single runs** and varied by ~±15 %. Replaced with medians of 3 runs. -8. **Earlier tier tables used 4 threads; `bench.sh` defaults to 16.** The tier tables are now labeled with 16 threads; the earlier 4-thread numbers are superseded. -9. **Row-parallel was inactive for Flint.** Added the gated (gate/up/down) path so v2 applies to Flint; verified active and numerically equivalent to v1 (see checks). +1. **Watchdog threshold never fired (found in this pass, most important):** `tools/runmem.py` compared `VmRSS` (kB) with the threshold in GiB × 1024³ (bytes), so no run could ever be killed. Every earlier "PASS, under threshold" and "never over threshold" statement is void. The peak RSS values were correctly measured. Fixed (threshold × 1024² kB) and verified: native at 2 GiB is now killed at 3.16 GiB. All tier results were re-run with enforcement; the old logs are in `results/logs/superseded_runmem_threshold_bug/`. +2. **`swiglu_limit` was dropped:** `sdq_build_moe` ignored its argument and the op always used 0. Fixed. This checkpoint has limit 0, so the numbers are unaffected. +3. **SwiGLU clamp mismatch:** the pager clamped `silu(gate)` to `[-limit, limit]`; upstream clamps only the upper bound. Fixed to match upstream. +4. **Bad debug hook (debug path only):** `sdq_maybe_dump_tensor` (enabled by `SDQ_MOE_OUT_DUMP`) wrote the tensor to a file but never copied it into its output, so downstream nodes read uninitialized memory. Fixed; the production path never used this node. +5. **Published patch was stale** at several points. Re-generated and re-uploaded after each change; the current patch is `b8a4e77…`. +6. **Smoke command in the model card was not the one that was run.** The card shows the executed command. +7. **`tools/runmem_cgroup.py` and the cgroup claim were presented as usable.** Labeled untested. +8. **Earlier memory-tier throughput came from single runs** and varied by ~±15 %. Replaced with medians of 3 runs. +9. **Earlier tier tables used 4 threads; `bench.sh` defaults to 16.** Tier tables now state 16 threads; the earlier 4-thread numbers are superseded. +10. **Row-parallel was inactive for Flint.** Added the gated path; verified active and numerically equivalent to v1. +11. **The forced-row-parallel speed gain (9–14 %) was not reproducible.** It came from the unenforced runs. With enforced runs the medians are within noise of v1 and v2 at all tiers. Retracted. ## Known limits and open items - EPQ output is not bit-identical to native llama.cpp. Float summation order differs, and 8-bit activation quantization amplifies the difference through the layers. Native itself shows a KL divergence of the same magnitude under a ubatch change. No functional mismatch was found. - Greedy identity is fragile: at 16 threads both v1 and v2 diverge from native after 12 characters. -- The memory tiers use an RSS watchdog that samples every 20 ms and counts file-backed pages as reported by `/proc`. They are not kernel-enforced limits. A true cgroup-enforced test requires a host with a writable cgroup v2 hierarchy and systemd (`tools/runmem_cgroup.py`, untested). -- v2 as designed activates only with a full arena (32 GiB tier). At 2–8 GiB it runs the same mode C path as v1 (0 row-parallel ops); at 16 GiB 12 of 516 ops are row-parallel. Forced row-parallel (`SDQ_ROWPAR_MAX_MISS`) is an ablation: its generation gains (about 9–14 %) are at the edge of run-to-run noise. +- The memory tiers use an RSS watchdog that samples every 20 ms and counts file-backed pages as reported by `/proc`. It is not a kernel-enforced limit: RSS may exceed the threshold between samples. A true cgroup-enforced test requires a host with a writable cgroup v2 hierarchy and systemd (`tools/runmem_cgroup.py`, untested). +- Native cannot run under 24.3 GiB with this watchdog; v1 and v2 need a 128 MiB arena to pass the 2 GiB tier. +- Throughput in the tier runs is noisy (one v1 4 GiB run gave 2.6 t/s against a median of 4.7). Treat speed differences under about 15 % as unresolved. - Accuracy tests use about 1,000 scored tokens from one text sample. They do not cover long-context behavior or other prompts beyond the greedy checks. - The v2 prompt-phase speedup is likely due to preload removing SSD reads during prefill; this was not isolated by a separate measurement. @@ -63,7 +67,8 @@ Versions: **v1** = expert paging, mode C (`SDQ_PRELOAD=0 SDQ_ROW_PARALLEL=0`). * `results/logs/{native,v1,v2}/` hold the per-version logs; `results/logs/common/` holds the corpus and summaries: - Prefill accuracy: `v1/ppl_prefill_ub64_t8.log`, `v1/kl_prefill_ub64_t8_vs_native.log`, `v2/ppl_prefill_ub64_t8.log`, `v2/kl_prefill_ub64_t8_vs_native.log`, `native/ppl_kl_base_prefill_ub64_t8.log`, `native/kl_ub16_vs_native_ub64.log` -- Decode accuracy: `v1/kl_decode_ub1_t16_vs_native.log`, `v2/kl_decode_ub1_t16_vs_native.log`, `v2/kl_decode_ub1_t16_vs_v1.log`, `native/ppl_kl_base_decode_ub1_t16.log`, `native/kl_ub64_vs_native_ub1_t16.log` +- Decode accuracy: `v1/kl_decode_ub1_t16_vs_native.log`, `v1/kl_decode_ub1_t16_modec_1g_vs_native.log`, `v2/kl_decode_ub1_t16_vs_native.log`, `v2/kl_decode_ub1_t16_vs_v1.log`, `v2/kl_decode_ub1_t16_forced_rowpar_1g_vs_native.log`, `native/ppl_kl_base_decode_ub1_t16.log`, `native/kl_ub64_vs_native_ub1_t16.log` - Greedy: `native/greedy_t8.txt`, `native/greedy_t16.txt`, `v1/greedy_t8.txt`, `v1/greedy_t16.txt`, `v2/greedy_t8.txt`, `v2/greedy_t16.txt` -- Throughput: `*/perf_c512_*.log`, `common/perf_c512_summary.txt` -- Memory: `v1/mem_v1_*.log`, `v2/v2_32g_run*.log`, `common/mem_summary.txt` +- Throughput (not enforced): `*/perf_c512_*.log`, `common/perf_c512_summary.txt` +- Memory, enforced: `native/mem_enforced_*.log`, `v1/mem_enforced_*.log` (including `_2g_arena128_`), `v2/mem_enforced_*.log` (default and forced, including `_2g_arena128_`), summaries `common/mem_enforced_summary.txt` and `common/mem_enforced_2g_arena128_summary.txt` +- Superseded (threshold bug, do not use for pass/fail): `superseded_runmem_threshold_bug/` diff --git a/tools/runmem.py b/tools/runmem.py index 765be84aa5d6574406e7b6e910d0d5be76d194bf..9d5e96c47dc0ab5bd173edfc119c46b1c4898c3c 100644 --- a/tools/runmem.py +++ b/tools/runmem.py @@ -52,7 +52,8 @@ def main(): pass peak_rss = max(peak_rss, rss) peak_swap = max(peak_swap, swap) - if args.rss_limit_gib is not None and rss > args.rss_limit_gib * (1024 ** 3): + # VmRSS 以 kB 為單位，門檻須換算成 kB（1 GiB = 1024 * 1024 kB） + if args.rss_limit_gib is not None and rss > args.rss_limit_gib * (1024 ** 2): exceeded = True process.send_signal(signal.SIGKILL) break