MOSS-VL-Realtime-NF4 / QUANTIZATION.md
CCCCyx's picture
Add files using upload-large-folder tool
2e600c2 verified
|
Raw
History Blame Contribute Delete
259 Bytes

MOSS-VL Streaming quantized checkpoint

  • Variant: w4_keep_ends
  • Method: bitsandbytes NF4 W4A16 keep-first-last-4
  • Compute dtype: bfloat16
  • Quantized Linear modules: 240
  • Vision tower, cross-attention, embeddings, lm_head and norms remain BF16.