models/Megatron/docs/source/api-guide/models.gpt.rst

models.gpt package
==================
This is the implementation of the popular GPT model. It supports several features like model parallelization (Tensor Parallel, Pipeline Parallel, Data Parallel) , mixture of experts, FP8 , Distributed optimizer etc. We are constantly adding new features. So be on the lookout or raise an issue if you want to have something added. 

Submodules
----------

models.gpt.gpt\_model module
----------------------------

.. automodule:: core.models.gpt.gpt_model
   :members:
   :undoc-members:
   :show-inheritance:

Module contents
---------------

.. automodule:: core.models.gpt
   :members:
   :undoc-members:
   :show-inheritance: