Instructions to use Composio/Mixtral-TensorRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Composio/Mixtral-TensorRT with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Composio/Mixtral-TensorRT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download .gitattributes from Composio/Mixtral-TensorRT: direct link, hf CLI and curl.
- Browser
- Download file 1.76 kB
-
https://huggingface.co/Composio/Mixtral-TensorRT/resolve/main/.gitattributes
- Command line
-
hf download hf://Composio/Mixtral-TensorRT/.gitattributes
-
curl -L -o .gitattributes https://huggingface.co/Composio/Mixtral-TensorRT/resolve/main/.gitattributes
1.76 kB
| llama_float16_tp8_rank2.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank3.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank4.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank5.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank6.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank7.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank0.engine filter=lfs diff=lfs merge=lfs -text | |
| llama_float16_tp8_rank1.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank0.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank1.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank2.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank3.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank4.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank5.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank6.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16/llama_bfloat16_tp8_rank7.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16_w2/llama_bfloat16_tp2_rank0.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_bf16_w2/llama_bfloat16_tp2_rank1.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_w2/llama_float16_tp2_rank0.engine filter=lfs diff=lfs merge=lfs -text | |
| TP_w2/llama_float16_tp2_rank1.engine filter=lfs diff=lfs merge=lfs -text | |
| PP_w2/llama_float16_tp1_pp2_rank0.engine filter=lfs diff=lfs merge=lfs -text | |
| PP_w2/llama_float16_tp1_pp2_rank1.engine filter=lfs diff=lfs merge=lfs -text | |
| PP_bf16_w2/llama_bfloat16_tp2_rank0.engine filter=lfs diff=lfs merge=lfs -text | |
| PP_bf16_w2/llama_bfloat16_tp2_rank1.engine filter=lfs diff=lfs merge=lfs -text | |