Instructions to use zai-org/chatglm2-6b-int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zai-org/chatglm2-6b-int4 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("zai-org/chatglm2-6b-int4", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
linux环境部署cpu模式
#19
by langya2 - opened
linux环境部署cpu模式使用时编译报错,始终编译不通过
Compile parallel cpu kernel gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/model/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/model/quantization_kernels_parallel.so failed.
Compile cpu kernel gcc -O3 -fPIC -std=c99 /root/.cache/huggingface/modules/transformers_modules/model/quantization_kernels.c -shared -o /root/.cache/huggingface/modules/transformers_modules/model/quantization_kernels.so failed.