accelerate/pr_4021/en/llms.txt

# Accelerate

## Docs

- [Accelerate](https://huggingface.co/docs/accelerate/pr_4021/index.md)
- [Quicktour](https://huggingface.co/docs/accelerate/pr_4021/quicktour.md)
- [Distributed inference](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/distributed_inference.md)
- [Profiler](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/profiler.md)
- [Intel Gaudi](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/gaudi.md)
- [Fully Sharded Data Parallel](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/fsdp.md)
- [Start Here!](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/explore.md)
- [Experiment trackers](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/tracking.md)
- [DDP Communication Hooks](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/ddp_comm_hook.md)
- [Using multiple models with DeepSpeed](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/deepspeed_multiple_model.md)
- [Low Precision Training Methods](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/low_precision_training.md)
- [Model memory estimator](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/model_size_estimator.md)
- [Accelerated PyTorch Training on Mac](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/mps.md)
- [Example Zoo](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/training_zoo.md)
- [Compilation](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/compilation.md)
- [Megatron-LM](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/megatron_lm.md)
- [Using Local SGD with Accelerate](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/local_sgd.md)
- [DeepSpeed](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/deepspeed.md)
- [Performing gradient accumulation with Accelerate](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/gradient_accumulation.md)
- [Big Model Inference](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/big_modeling.md)
- [Model quantization](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/quantization.md)
- [Amazon SageMaker](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/sagemaker.md)
- [Training on Intel CPU](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/intel_cpu.md)
- [Checkpointing](https://huggingface.co/docs/accelerate/pr_4021/usage_guides/checkpoint.md)
- [Gradient synchronization](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/gradient_synchronization.md)
- [Loading big models into memory](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/big_model_inference.md)
- [Training on TPUs](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/training_tpu.md)
- [Comparing performance across distributed setups](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/performance.md)
- [Executing and deferring jobs](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/deferring_execution.md)
- [Low precision training methods](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/low_precision_training.md)
- [Accelerate's internal mechanisms](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/internal_mechanism.md)
- [FSDP vs DeepSpeed](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/fsdp_and_deepspeed.md)
- [Sequence parallel in 🤗`accelerate`](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/sequence_parallelism.md)
- [FSDP1 vs FSDP2](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/fsdp1_vs_fsdp2.md)
- [Context Parallel in 🤗`accelerate`](https://huggingface.co/docs/accelerate/pr_4021/concept_guides/context_parallelism.md)
- [DataLoaders, Optimizers, and Schedulers](https://huggingface.co/docs/accelerate/pr_4021/package_reference/torch_wrappers.md)
- [Launchers](https://huggingface.co/docs/accelerate/pr_4021/package_reference/launchers.md)
- [Stateful Classes](https://huggingface.co/docs/accelerate/pr_4021/package_reference/state.md)
- [Fully Sharded Data Parallel utilities](https://huggingface.co/docs/accelerate/pr_4021/package_reference/fsdp.md)
- [Experiment Trackers](https://huggingface.co/docs/accelerate/pr_4021/package_reference/tracking.md)
- [The Command Line](https://huggingface.co/docs/accelerate/pr_4021/package_reference/cli.md)
- [Megatron-LM utilities](https://huggingface.co/docs/accelerate/pr_4021/package_reference/megatron_lm.md)
- [DeepSpeed utilities](https://huggingface.co/docs/accelerate/pr_4021/package_reference/deepspeed.md)
- [Pipeline parallelism](https://huggingface.co/docs/accelerate/pr_4021/package_reference/inference.md)
- [Working with large models](https://huggingface.co/docs/accelerate/pr_4021/package_reference/big_modeling.md)
- [Accelerator](https://huggingface.co/docs/accelerate/pr_4021/package_reference/accelerator.md)
- [FP8](https://huggingface.co/docs/accelerate/pr_4021/package_reference/fp8.md)
- [Kwargs handlers](https://huggingface.co/docs/accelerate/pr_4021/package_reference/kwargs.md)
- [Logging[[accelerate.logging.get_logger]]](https://huggingface.co/docs/accelerate/pr_4021/package_reference/logging.md)
- [Utility functions and classes](https://huggingface.co/docs/accelerate/pr_4021/package_reference/utilities.md)
- [TPU training](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/tpu.md)
- [Overview](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/overview.md)
- [Installation](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/install.md)
- [Launching distributed training from Jupyter Notebooks](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/notebook.md)
- [Troubleshoot](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/troubleshooting.md)
- [Add Accelerate to your code](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/migration.md)
- [Execution process](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/execution.md)
- [Launching Accelerate scripts](https://huggingface.co/docs/accelerate/pr_4021/basic_tutorials/launch.md)