Transformers
Safetensors
English
llama4
llama4-text
llama4-moe
Mixture of Experts
mixture-of-experts
causal-lm
tinystories
tiny-model
validation
debug-model
Instructions to use shibatch/tinyllama4moe2m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tinyllama4moe2m with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tinyllama4moe2m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -286,36 +286,6 @@ with torch.no_grad():
|
|
| 286 |
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
|
| 287 |
```
|
| 288 |
|
| 289 |
-
If the model files are stored at the repository root, remove `subfolder="hf"`:
|
| 290 |
-
|
| 291 |
-
```python
|
| 292 |
-
import torch
|
| 293 |
-
from transformers import PreTrainedTokenizerFast, Llama4ForCausalLM
|
| 294 |
-
|
| 295 |
-
repo = "shibatch/tinyllama4moe2m"
|
| 296 |
-
|
| 297 |
-
tokenizer = PreTrainedTokenizerFast.from_pretrained(repo)
|
| 298 |
-
model = Llama4ForCausalLM.from_pretrained(
|
| 299 |
-
repo,
|
| 300 |
-
torch_dtype=torch.float32,
|
| 301 |
-
)
|
| 302 |
-
model.eval()
|
| 303 |
-
|
| 304 |
-
prompt = "Once upon"
|
| 305 |
-
inputs = tokenizer(prompt, return_tensors="pt")
|
| 306 |
-
|
| 307 |
-
with torch.no_grad():
|
| 308 |
-
output_ids = model.generate(
|
| 309 |
-
**inputs,
|
| 310 |
-
max_new_tokens=100,
|
| 311 |
-
do_sample=False,
|
| 312 |
-
pad_token_id=tokenizer.pad_token_id,
|
| 313 |
-
eos_token_id=tokenizer.eos_token_id,
|
| 314 |
-
)
|
| 315 |
-
|
| 316 |
-
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
|
| 317 |
-
```
|
| 318 |
-
|
| 319 |
## Loading requirements
|
| 320 |
|
| 321 |
This checkpoint requires a Transformers version that supports Llama4 and Llama4 MoE.
|
|
|
|
| 286 |
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
|
| 287 |
```
|
| 288 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 289 |
## Loading requirements
|
| 290 |
|
| 291 |
This checkpoint requires a Transformers version that supports Llama4 and Llama4 MoE.
|