| --- |
| |
| |
| {} |
| --- |
| |
| # Model Card for Phi 2 SlimOrca |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
| Phi 2 finetuned on SlimOrca-Dedup. This model was trained with the goal of giving Phi 2 the ablity to generate the EOS token together with being capable of doing beam search. It can also follow custom system prompts as shown in the example below. |
|
|
| ## Model Details |
|
|
| ## How to Get Started with the Model |
|
|
| ```python |
| import torch |
| import transformers |
| |
| model = transformers.AutoModelForCausalLM.from_pretrained( |
| "miguelcarv/phi-2-slimorca", |
| trust_remote_code=True, |
| ).to('cuda') |
| tokenizer = transformers.AutoTokenizer.from_pretrained("microsoft/phi-2") |
| |
| |
| SYSTEM_PROMPT = "You are an AI assistant. You will be given a task. You must generate a short and concise answer." |
| input_text = f"""{SYSTEM_PROMPT} |
| |
| Instruction: Give me the first 5 prime numbers and explain what prime numbers are. |
| Output:""" |
| |
| with torch.no_grad(): |
| outputs = model.generate( |
| tokenizer(input_text, return_tensors="pt")['input_ids'].to('cuda'), |
| max_length=1024, |
| num_beams = 3, |
| eos_token_id = tokenizer.eos_token_id |
| ) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Training Details |
|
|
| - Trained for one epoch on SlimOrca-Dedup |
| - Learning rate: 1e-5 |
| - Cosine learning rate decay to 0 |
| - Optimizer: AdamW |
| - Batch size: 256 |
| - Trained with mixed-precision bfloat16 |
|
|
