zai-org
/

chatglm2-6b-int4

Model card Files Files and versions

INT4 model shows bad perf than FP32 on Intel CPU,why?

#13

by Sakura10151 - opened Jul 19, 2023

int4 model log shows it will consume hundreds of times more time than the fp32 model on SelfAttention

Me too, why? https://github.com/THUDM/ChatGLM2-6B#cpu-%E9%83%A8%E7%BD%B2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment