Instructions to use tencent/Hunyuan-1.8B-Instruct-GPTQ-Int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/Hunyuan-1.8B-Instruct-GPTQ-Int4 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("tencent/Hunyuan-1.8B-Instruct-GPTQ-Int4", dtype="auto") - Notebooks
- Google Colab
- Kaggle
transformers-4.37.2 & optimum 是不支持`hunyuan_v1_dense`
#3
by YouLei0106 - opened
Traceback (most recent call last):
File "anaconda3\envs\python3\lib\site-packages\transformers\models\auto\configuration_auto.py", line 1265, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "anaconda3\envs\python3\lib\site-packages\transformers\models\auto\configuration_auto.py", line 960, in getitem
raise KeyError(key)
KeyError: 'hunyuan_v1_dense'
YouLei0106 changed discussion title from transformers-4.37.2 是不支持`hunyuan_v1_dense` to transformers-4.37.2 & optimum 是不支持`hunyuan_v1_dense`