steampunque
/

Qwen3-Coder-Next-MP-GGUF

Qwen3 Coder Next

4-bit precision

mixed precision

Model card Files Files and versions

steampunque commited on Feb 18

Commit

b0819eb

·

verified ·

1 Parent(s): ce6c4f5

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -62,7 +62,7 @@ full CPU expert offload.
 Rough performance metrics on a 9900k (128G RAM) and 4070 (12G VRAM)
 CPU EXP OFFLOAD | QKV  | Context size| gen rate  |  ot config
-------------|------|-------------|-----------|------------
 all         | F16  |   256K      |  20.1     | OT="-ot exps=CPU -ngl 99"
 4-48        | F16  |   256K      |  8        | OT="-ot blk\\.[4-9]\|1[0-9]\|2[0-9]\|3[0-9]\|4[0-7].*exps=CPU -ngl 99"
 7-48        | F16  |   128K      |  10       | OT="-ot blk\\.[7-9]\|1[0-9]\|2[0-9]\|3[0-9]\|4[0-7].*exps=CPU -ngl 99"

 Rough performance metrics on a 9900k (128G RAM) and 4070 (12G VRAM)
 CPU EXP OFFLOAD | QKV  | Context size| gen rate  |  ot config
+----------------|------|-------------|-----------|------------
 all         | F16  |   256K      |  20.1     | OT="-ot exps=CPU -ngl 99"
 4-48        | F16  |   256K      |  8        | OT="-ot blk\\.[4-9]\|1[0-9]\|2[0-9]\|3[0-9]\|4[0-7].*exps=CPU -ngl 99"
 7-48        | F16  |   128K      |  10       | OT="-ot blk\\.[7-9]\|1[0-9]\|2[0-9]\|3[0-9]\|4[0-7].*exps=CPU -ngl 99"