Instructions to use MoxoffSrL/Moxoff-Phi3Mini-KTO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MoxoffSrL/Moxoff-Phi3Mini-KTO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MoxoffSrL/Moxoff-Phi3Mini-KTO", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MoxoffSrL/Moxoff-Phi3Mini-KTO", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("MoxoffSrL/Moxoff-Phi3Mini-KTO", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use MoxoffSrL/Moxoff-Phi3Mini-KTO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MoxoffSrL/Moxoff-Phi3Mini-KTO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoxoffSrL/Moxoff-Phi3Mini-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MoxoffSrL/Moxoff-Phi3Mini-KTO

SGLang

How to use MoxoffSrL/Moxoff-Phi3Mini-KTO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MoxoffSrL/Moxoff-Phi3Mini-KTO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoxoffSrL/Moxoff-Phi3Mini-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MoxoffSrL/Moxoff-Phi3Mini-KTO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoxoffSrL/Moxoff-Phi3Mini-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use MoxoffSrL/Moxoff-Phi3Mini-KTO with Docker Model Runner:
```
docker model run hf.co/MoxoffSrL/Moxoff-Phi3Mini-KTO
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Information

Moxoff-Phi3Mini-KTO is an updated version of Phi-3-mini-128k-instruct, aligned with KTO and QLora.

It's trained on distilabel-intel-orca-kto.

Evaluation

We evaluated the model using the same test sets as used for the Open LLM Leaderboard

hellaswag acc_norm	arc_challenge acc_norm	m_mmlu 5-shot acc	Average
0.7915	0.5606	0.6939	0.682

Usage

Be sure to install these dependencies before running the program

!pip install transformers torch sentencepiece

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cpu" # if you want to use the gpu make sure to have cuda toolkit installed and change this to "cuda"

model = AutoModelForCausalLM.from_pretrained("MoxoffSpA/Moxoff-Phi3Mini-KTO")
tokenizer = AutoTokenizer.from_pretrained("MoxoffSpA/Moxoff-Phi3Mini-KTO")

question = """Quanto è alta la torre di Pisa?"""
context = """
La Torre di Pisa è un campanile del XII secolo, famoso per la sua inclinazione. Alta circa 56 metri.
"""

prompt = f"Domanda: {question}, contesto: {context}"

messages = [
    {"role": "user", "content": prompt}
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(
    model_inputs, # The input to the model
    max_new_tokens=128, # Limiting the maximum number of new tokens generated
    do_sample=True, # Enabling sampling to introduce randomness in the generation
    temperature=0.1, # Setting temperature to control the randomness, lower values make it more deterministic
    top_p=0.95, # Using nucleus sampling with top-p filtering for more coherent generation       
    eos_token_id=tokenizer.eos_token_id # Specifying the token that indicates the end of a sequence
)

decoded_output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
trimmed_output = decoded_output.strip()
print(trimmed_output)

Bias, Risks and Limitations

Moxoff-Phi3Mini-KTO has not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model, however it is likely to have included a mix of Web data and technical sources like books and code.