MOSS-VL-Realtime-NF4 / QUANTIZATION.md
CCCCyx's picture
Add files using upload-large-folder tool
2e600c2 verified
|
Raw
History Blame Contribute Delete
259 Bytes
# MOSS-VL Streaming quantized checkpoint
- Variant: `w4_keep_ends`
- Method: `bitsandbytes NF4 W4A16 keep-first-last-4`
- Compute dtype: `bfloat16`
- Quantized Linear modules: `240`
- Vision tower, cross-attention, embeddings, lm_head and norms remain BF16.