Instructions to use KexuanShi/Megatron-LM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NeMo
How to use KexuanShi/Megatron-LM with NeMo:
# tag did not correspond to a valid NeMo domain.
- Notebooks
- Google Colab
- Kaggle
Megatron Discussions
This directory contains in-depth guides, tutorials, and discussions about optimizing and using Megatron for various use cases.
Available Guides
Training Guides
-
A practical guide to enable Megatron-FSDP training, including a quick-start example for DeepSeek-V3, required and recommended configurations, and instructions for checkpoint conversion from torch_dist to fsdp_dtensor.
Contributing
If you'd like to contribute a guide or tutorial, please follow this structure:
- Create a new directory:
docs/discussions/your-guide-name/ - Add your main guide:
docs/discussions/your-guide-name/your-guide-name.md - Create an images directory:
docs/discussions/your-guide-name/images/ - Update this README.md with a link to your guide
Each guide should be self-contained with its own images and supporting files.