shibatch commited on
Commit
b23d41d
·
verified ·
1 Parent(s): e21f2ba

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +0 -30
README.md CHANGED
@@ -286,36 +286,6 @@ with torch.no_grad():
286
  print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
287
  ```
288
 
289
- If the model files are stored at the repository root, remove `subfolder="hf"`:
290
-
291
- ```python
292
- import torch
293
- from transformers import PreTrainedTokenizerFast, Llama4ForCausalLM
294
-
295
- repo = "shibatch/tinyllama4moe2m"
296
-
297
- tokenizer = PreTrainedTokenizerFast.from_pretrained(repo)
298
- model = Llama4ForCausalLM.from_pretrained(
299
- repo,
300
- torch_dtype=torch.float32,
301
- )
302
- model.eval()
303
-
304
- prompt = "Once upon"
305
- inputs = tokenizer(prompt, return_tensors="pt")
306
-
307
- with torch.no_grad():
308
- output_ids = model.generate(
309
- **inputs,
310
- max_new_tokens=100,
311
- do_sample=False,
312
- pad_token_id=tokenizer.pad_token_id,
313
- eos_token_id=tokenizer.eos_token_id,
314
- )
315
-
316
- print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
317
- ```
318
-
319
  ## Loading requirements
320
 
321
  This checkpoint requires a Transformers version that supports Llama4 and Llama4 MoE.
 
286
  print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
287
  ```
288
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
289
  ## Loading requirements
290
 
291
  This checkpoint requires a Transformers version that supports Llama4 and Llama4 MoE.