| | --- |
| | language: |
| | - zh |
| | - en |
| | tags: |
| | - glm |
| | - chatglm |
| | - thudm |
| | --- |
| | |
| | # ChatGLM2 6b int8 量化模型 |
| |
|
| | 详情参考 [K024/chatglm-q](https://github.com/K024/chatglm-q)。 |
| |
|
| | See [K024/chatglm-q](https://github.com/K024/chatglm-q) for more details. |
| |
|
| | ```python |
| | import torch |
| | from chatglm_q.decoder import ChatGLMDecoder, chat_template |
| | |
| | device = torch.device("cuda") |
| | decoder = ChatGLMDecoder.from_pretrained("K024/chatglm2-6b-int8", device=device) |
| | |
| | prompt = chat_template([], "我是谁？") |
| | for text in decoder.generate(prompt): |
| | print(text) |
| | ``` |
| |
|
| | 模型权重按 ChatGLM2-6b 许可发布，见 [MODEL LICENSE](https://huggingface.co/THUDM/chatglm2-6b/blob/main/MODEL_LICENSE)。 |
| |
|
| | Model weights are released under the same license as ChatGLM2-6b, see [MODEL LICENSE](https://huggingface.co/THUDM/chatglm2-6b/blob/main/MODEL_LICENSE). |
| |
|
