Instructions to use zai-org/chatglm-6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zai-org/chatglm-6b with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("zai-org/chatglm-6b", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Fix LogitsProcessor using slim checkpoint
Browse filesLogitsProcessor token id should match the tokenizer
- modeling_chatglm.py +1 -1
modeling_chatglm.py
CHANGED
|
@@ -55,7 +55,7 @@ class InvalidScoreLogitsProcessor(LogitsProcessor):
|
|
| 55 |
def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
|
| 56 |
if torch.isnan(scores).any() or torch.isinf(scores).any():
|
| 57 |
scores.zero_()
|
| 58 |
-
scores[...,
|
| 59 |
return scores
|
| 60 |
|
| 61 |
|
|
|
|
| 55 |
def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
|
| 56 |
if torch.isnan(scores).any() or torch.isinf(scores).any():
|
| 57 |
scores.zero_()
|
| 58 |
+
scores[..., 5] = 5e4
|
| 59 |
return scores
|
| 60 |
|
| 61 |
|