Hugging Face's logo

KexuanShi
/
Megatron-LM

NeMo
Model card Files
xet
 Community
Megatron-LM / docs /user-guide /features /moe.md
KexuanShi's picture
KexuanShi
Upload folder using huggingface_hub
88e6849 verified
preview code
|
Raw
History Blame Contribute Delete
189 Bytes

Mixture of Experts 

:maxdepth: 1
:caption: MoE Features

multi_token_prediction
multi_latent_attention