2.41 kB

Ctrl+K

1 contributor

gemma-4-E4B-it W4A16 (full-precision lm_head), calibrated on our own CLI/tool-call logs, + Gemma-4 MTP drafter for speculative decoding. ~130 tok/s single-stream on one RTX 4060 Ti, stock vLLM 0.26.0, correct tool calls, no vocab pruning.

c25f58d verified 1 day ago

launch.sh

1.64 kB
gemma-4-E4B-it W4A16 (full-precision lm_head), calibrated on our own CLI/tool-call logs, + Gemma-4 MTP drafter for speculative decoding. ~130 tok/s single-stream on one RTX 4060 Ti, stock vLLM 0.26.0, correct tool calls, no vocab pruning. 1 day ago
requirements.txt

776 Bytes
gemma-4-E4B-it W4A16 (full-precision lm_head), calibrated on our own CLI/tool-call logs, + Gemma-4 MTP drafter for speculative decoding. ~130 tok/s single-stream on one RTX 4060 Ti, stock vLLM 0.26.0, correct tool calls, no vocab pruning. 1 day ago