File size: 1,072 Bytes
45194e5 e602790 ba21751 45194e5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | ---
license: mit
language:
- en
---
A Moe model built on top of microsoft/phi-2, g-ronimo/phi-2-OpenHermes-2.5 and mlx-community/phi-2-dpo-7k, random init gates weights
## Example
```
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch
DEV = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model_name_or_path = "mzbac/phi2-2x3"
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
)
model.to(DEV)
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
prompt = "Instruct: how backpropagation works.\nOutput:"
print("\n\n*** Generate:")
inputs = tokenizer.encode(prompt, return_tensors="pt").to(DEV)
generate_kwargs = dict(
input_ids=inputs,
temperature=0.3,
max_new_tokens=500,
do_sample=True,
)
outputs = model.generate(**generate_kwargs)
print(tokenizer.decode(outputs[0]))
``` |