File size: 1,265 Bytes
5974fd0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | ## Transformers / safetensors support
This model can now be loaded with `AutoModelForCausalLM` instead of the
manual pickle-loading workflow, and weights are available as
`model.safetensors`.
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True, dtype=torch.float32,
)
tokenizer = AutoTokenizer.from_pretrained("IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True)
model.eval()
inputs = tokenizer("Once upon a time, there was a", return_tensors="pt")
out = model.generate(
**inputs, max_new_tokens=200, do_sample=True,
temperature=0.8, top_k=50, pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```
`trust_remote_code=True` is required (custom architecture: RoPE + SwiGLU +
RMSNorm dense decoder). The original `ckpt_final.pt` pickle checkpoint and
`model/` architecture source remain in this repo unchanged for backwards
compatibility.
**Note on batch generation:** use left-padding
(`tokenizer.padding_side = "left"`) — the model doesn't use an explicit
attention mask over padded positions, so right-padding within a batch will
give incorrect results.
|