| # Mixture of Experts | |
| ```{toctree} | |
| :maxdepth: 1 | |
| :caption: MoE Features | |
| multi_token_prediction | |
| multi_latent_attention | |
| ``` | |
| ```{include} ../../../megatron/core/transformer/moe/README.md | |
| ``` | |
| # Mixture of Experts | |
| ```{toctree} | |
| :maxdepth: 1 | |
| :caption: MoE Features | |
| multi_token_prediction | |
| multi_latent_attention | |
| ``` | |
| ```{include} ../../../megatron/core/transformer/moe/README.md | |
| ``` | |