# tensor_parallel package

This package contains an implementation for tensor parallelism in transformer
models (see [Megatron-LM: Training Multi-Billion Parameter Language Models
Using Model Parallelism](https://arxiv.org/abs/1909.08053) and [Reducing
Activation Recomputation in Large Transformer Models](https://arxiv.org/abs/2205.05198)
for details).