Instructions to use KexuanShi/Megatron-LM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NeMo
How to use KexuanShi/Megatron-LM with NeMo:
# tag did not correspond to a valid NeMo domain.
- Notebooks
- Google Colab
- Kaggle
Ctrl+K
- gpt3_15b_8t_release
- gpt3_15b_8t_release_sm
- gpt3_7b_tp1_pp4_memory_speed
- gpt3_7b_tp4_pp1_memory_speed
- gpt3_mcore_reruns_disable
- gpt3_mcore_reruns_enable
- gpt3_mcore_reruns_persistent_1
- gpt3_mcore_reruns_persistent_2
- gpt3_mcore_reruns_reshard
- gpt3_mcore_reruns_resume
- gpt3_mcore_reruns_resume_check_grads
- gpt3_mcore_reruns_transient
- gpt3_mcore_te_tp1_pp1_dist_optimizer_fim_dataset
- gpt3_mcore_te_tp1_pp1_dist_optimizer_no_mmap_bin_files
- gpt3_mcore_te_tp1_pp1_frozen_resume_torch_dist_dist_optimizer
- gpt3_mcore_te_tp1_pp1_resume_torch_dist_dist_optimizer
- gpt3_mcore_te_tp1_pp1_resume_torch_dist_dist_optimizer_no_mmap_bin_files
- gpt3_mcore_te_tp1_pp1_resume_torch_dist_uniform_full_recompute
- gpt3_mcore_te_tp1_pp1_uniform_full_recompute
- gpt3_mcore_te_tp1_pp2_cp4_a2a_p2p_nondeterministic
- gpt3_mcore_te_tp1_pp2_resume_torch_dist_cp4_a2a_p2p_nondeterministic
- gpt3_mcore_te_tp1_pp2_resume_torch_dist_rope_embeddings
- gpt3_mcore_te_tp1_pp2_resume_torch_dist_rope_embeddings_interleaved_no_fusion
- gpt3_mcore_te_tp1_pp2_rope_embeddings
- gpt3_mcore_te_tp1_pp2_rope_embeddings_interleaved_no_fusion
- gpt3_mcore_te_tp1_pp4_disable_bias_linear
- gpt3_mcore_te_tp1_pp4_frozen_resume_torch_dist_swiglu
- gpt3_mcore_te_tp1_pp4_persistent_ckpt_disable_bias_linear
- gpt3_mcore_te_tp1_pp4_resume_torch_dist_disable_bias_linear
- gpt3_mcore_te_tp1_pp4_resume_torch_dist_persistent_disable_bias_linear
- gpt3_mcore_te_tp1_pp4_resume_torch_dist_sequence_parallel
- gpt3_mcore_te_tp1_pp4_resume_torch_dist_swiglu
- gpt3_mcore_te_tp1_pp4_resume_torch_dist_untie_embeddings_and_outputs
- gpt3_mcore_te_tp1_pp4_sequence_parallel
- gpt3_mcore_te_tp1_pp4_swiglu
- gpt3_mcore_te_tp1_pp4_untie_embeddings_and_outputs
- gpt3_mcore_te_tp1_pp4_vp1
- gpt3_mcore_te_tp1_pp4_vp1_calculate_per_token_loss
- gpt3_mcore_te_tp1_pp4_vp1_decoupled_lr
- gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce
- gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_param_gather
- gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_param_gather_overlap_optimizer
- gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_untied
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_decoupled_lr
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_dist
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_dist_calculate_per_token_loss
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce_param_gather
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce_untied
- gpt3_mcore_te_tp1_pp4_vp1_resume_torch_dist_tunable_overlap