File size: 356 Bytes
88e6849 | 1 2 3 4 5 6 7 8 9 | # tensor_parallel package
This package contains an implementation for tensor parallelism in transformer
models (see [Megatron-LM: Training Multi-Billion Parameter Language Models
Using Model Parallelism](https://arxiv.org/abs/1909.08053) and [Reducing
Activation Recomputation in Large Transformer Models](https://arxiv.org/abs/2205.05198)
for details).
|