Instructions to use ibm-ai-platform/llama3-70b-accelerator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ibm-ai-platform/llama3-70b-accelerator with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ibm-ai-platform/llama3-70b-accelerator", device_map="auto") - Notebooks
- Google Colab
- Kaggle
AWQ INT 4 Model
#3 opened over 1 year ago
by
ohyay12345
Some missing stuffs to run on tgi server
27
#2 opened about 2 years ago
by
yangxia20000