Speculative-decoding drafts - a JRamirez-UAB Collection

JRamirez-UAB 's Collections

Sysadmin agent models: our picks

27B Dense MTP Vision for 24 GB GPUs

Speculative-decoding drafts

9B Dense MTP Vision for 12–16 GB GPUs

35B-A3B MoE MTP Vision for 24 GB GPUs

125B-A6B MoE Flash Next MTP Vision for 48-96 GB GPUs

Helpers: decision and triage

Speculative-decoding drafts

updated 3 days ago

Draft models that speed up our models with speculative decoding. The output is the same as the target model's.

JRamirez-UAB/Qwen3.8-27B-DFlash2-W4A16

Text Generation • 2B • Updated 2 days ago • 688 • 1

Note DFlash2 draft for Qwen3.8-27B (W4A16 main, W8A16 branch). Tested on one and on two RTX 3090 (the 27B and the draft on the same GPU or on separate ones).