File size: 458 Bytes
d7c049e | 1 2 3 4 5 6 7 8 9 | # OpenMOSS-Team/MOSS-VL-Realtime-FP8
- Runtime: Transformers `online_generate` / `create_realtime_session`.
- Weights: compressed-tensors FP8 E4M3 on 252 self-attention/MLP Linear layers.
- Activations: FP8 per-token dynamic on the same 252 Linear inputs.
- BF16: cross-attention layers, vision encoder/merger, embeddings, norms and `lm_head`.
- KV cache: Transformers HQQ INT8, group size 64, residual length 128.
- Attention backend: `flash_attention_2`.
|