ereniko commited on
Commit
4491ba3
·
verified ·
1 Parent(s): 5974fd0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -3
README.md CHANGED
@@ -147,9 +147,43 @@ Custom byte-level BPE tokenizer trained from scratch on a sample of the pretrain
147
 
148
  ---
149
 
150
- ## Inference
151
 
152
- Here's a basic inference code you can run to immediately start using İvme-Conversate-v2-Base
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
153
 
154
  ```python
155
  import sys
@@ -234,7 +268,7 @@ You can check our other upcoming models on our organization card!
234
  ## Citation
235
 
236
  ```bibtex
237
- @misc{ivme-conversate-v2-24m,
238
  author = {IvmeLabs},
239
  title = {İvme-Conversate-v2-Base},
240
  year = {2026},
 
147
 
148
  ---
149
 
150
+ ## Inference
151
 
152
+ This model can now be loaded with `AutoModelForCausalLM` instead of the
153
+ manual pickle-loading workflow, and weights are available as
154
+ `model.safetensors`.
155
+
156
+ ```python
157
+ import torch
158
+ from transformers import AutoModelForCausalLM, AutoTokenizer
159
+
160
+ model = AutoModelForCausalLM.from_pretrained(
161
+ "IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True, dtype=torch.float32,
162
+ )
163
+ tokenizer = AutoTokenizer.from_pretrained("IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True)
164
+ model.eval()
165
+
166
+ inputs = tokenizer("Once upon a time, there was a", return_tensors="pt")
167
+ out = model.generate(
168
+ **inputs, max_new_tokens=200, do_sample=True,
169
+ temperature=0.8, top_k=50, pad_token_id=tokenizer.pad_token_id,
170
+ )
171
+ print(tokenizer.decode(out[0], skip_special_tokens=True))
172
+ ```
173
+
174
+ `trust_remote_code=True` is required (custom architecture: RoPE + SwiGLU +
175
+ RMSNorm dense decoder). The original `ckpt_final.pt` pickle checkpoint and
176
+ `model/` architecture source remain in this repo unchanged for backwards
177
+ compatibility.
178
+
179
+ **Note on batch generation:** use left-padding
180
+ (`tokenizer.padding_side = "left"`) the model doesn't use an explicit
181
+ attention mask over padded positions, so right-padding within a batch will
182
+ give incorrect results.
183
+
184
+ ## Legacy Inference
185
+
186
+ Here's a basic inference code in case you want to work with the pickle files.
187
 
188
  ```python
189
  import sys
 
268
  ## Citation
269
 
270
  ```bibtex
271
+ @misc{ivme-conversate-v2-Base,
272
  author = {IvmeLabs},
273
  title = {İvme-Conversate-v2-Base},
274
  year = {2026},