Instructions to use justinj92/dpoplatypus-phi2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use justinj92/dpoplatypus-phi2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="justinj92/dpoplatypus-phi2", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("justinj92/dpoplatypus-phi2", trust_remote_code=True, dtype="auto")

llama-cpp-python

How to use justinj92/dpoplatypus-phi2 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="justinj92/dpoplatypus-phi2",
	filename="phi2-platypus-dpo-f32.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use justinj92/dpoplatypus-phi2 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf justinj92/dpoplatypus-phi2:F32
# Run inference directly in the terminal:
llama-cli -hf justinj92/dpoplatypus-phi2:F32

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf justinj92/dpoplatypus-phi2:F32
# Run inference directly in the terminal:
llama-cli -hf justinj92/dpoplatypus-phi2:F32

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf justinj92/dpoplatypus-phi2:F32
# Run inference directly in the terminal:
./llama-cli -hf justinj92/dpoplatypus-phi2:F32

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf justinj92/dpoplatypus-phi2:F32
# Run inference directly in the terminal:
./build/bin/llama-cli -hf justinj92/dpoplatypus-phi2:F32

Use Docker

docker model run hf.co/justinj92/dpoplatypus-phi2:F32

LM Studio
Jan

vLLM

How to use justinj92/dpoplatypus-phi2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "justinj92/dpoplatypus-phi2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "justinj92/dpoplatypus-phi2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/justinj92/dpoplatypus-phi2:F32

SGLang

How to use justinj92/dpoplatypus-phi2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "justinj92/dpoplatypus-phi2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "justinj92/dpoplatypus-phi2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "justinj92/dpoplatypus-phi2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "justinj92/dpoplatypus-phi2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Ollama
How to use justinj92/dpoplatypus-phi2 with Ollama:
```
ollama run hf.co/justinj92/dpoplatypus-phi2:F32
```

Unsloth Studio new

How to use justinj92/dpoplatypus-phi2 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for justinj92/dpoplatypus-phi2 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for justinj92/dpoplatypus-phi2 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for justinj92/dpoplatypus-phi2 to start chatting

Docker Model Runner
How to use justinj92/dpoplatypus-phi2 with Docker Model Runner:
```
docker model run hf.co/justinj92/dpoplatypus-phi2:F32
```

Lemonade

How to use justinj92/dpoplatypus-phi2 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull justinj92/dpoplatypus-phi2:F32

Run and chat with the model

lemonade run user.dpoplatypus-phi2-F32

List all available models

lemonade list

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

See axolotl config

axolotl version: 0.3.0

base_model: justinj92/phi2-platypus
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
is_llama_derived_model: false
trust_remote_code: true

load_in_8bit: false
load_in_4bit: false
strict: false

rl: true
datasets:
  - path: Intel/orca_dpo_pairs
    split: train
    type: intel_apply_chatml
  - path: argilla/ultrafeedback-binarized-preferences
    split: train
    type: argilla_apply_chatml
dataset_prepared_path: ./dpoplatypus-phi2/last_run_prepared
val_set_size: 0.0
output_dir: ./dpoplatypus-phi2/
#'Wqkv', 'out_proj', 'fc2', 'linear', 'fc1'
adapter:
sequence_len: 2048
sample_packing: false
pad_to_sequence_len:

lora_r: 64
lora_alpha: 32
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:
lora_modules_to_save:
  - embd
  - lm_head
hub_model_id: justinj92/phi2-platypus-dpo


wandb_project: phi2-platypus-dpo
wandb_entity: justinjoy-5
wandb_watch:
wandb_run_id:
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 1
num_epochs: 1
optimizer: adamw_torch
adam_beta2: 0.95
adam_epsilion: 0.00001
lr_scheduler: cosine
max_grad_norm: 1.0
learning_rate: 0.00002

train_on_inputs: false
group_by_length: false
bf16: true
fp16: false
tf32: true

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 100
eval_steps:
evals_per_epoch: 4
saves_per_epoch: 2
eval_table_size:
debug:
deepspeed:
weight_decay: 0.1
fsdp:
fsdp_config:
resize_token_embeddings_to_32x: true
special_tokens:
  pad_token: "<|endoftext|>"

dpoplatypus-phi2

This model is a fine-tuned version of justinj92/phi2-platypus on the None dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 1
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
training_steps: 19120

Training results

Framework versions

Transformers 4.37.0.dev0
Pytorch 2.0.1+cu117
Datasets 2.16.1
Tokenizers 0.15.0

Downloads last month: -

Safetensors

Model size

3B params

Tensor type

F32

BF16

Model tree for justinj92/dpoplatypus-phi2

Base model

justinj92/phi2-platypus

Quantized

(1)

this model

justinj92
/

dpoplatypus-phi2

You need to agree to share your contact information to access this model

dpoplatypus-phi2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Model tree for justinj92/dpoplatypus-phi2

Datasets used to train justinj92/dpoplatypus-phi2

Spaces using justinj92/dpoplatypus-phi2 2