Instructions to use tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO")
model = AutoModelForCausalLM.from_pretrained("tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO

SGLang

How to use tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO with Docker Model Runner:
```
docker model run hf.co/tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO
```

UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO

This model is a fine-tuned version of tsavage68/UTI_L3_1000steps_1e5rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0094
Rewards/chosen: 3.5398
Rewards/rejected: -9.3115
Rewards/accuracies: 0.9900
Rewards/margins: 12.8514
Logps/rejected: -61.8926
Logps/chosen: -22.1453
Logits/rejected: -1.1592
Logits/chosen: -1.1419

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5446	0.3333	25	0.2409	0.7934	-0.6030	0.9800	1.3964	-44.4754	-27.6381	-1.1424	-1.1365
0.0009	0.6667	50	0.0261	2.3344	-5.4705	0.9800	7.8050	-54.2106	-24.5560	-1.1516	-1.1414
0.0001	1.0	75	0.0417	2.5058	-6.7400	0.9700	9.2458	-56.7494	-24.2133	-1.1557	-1.1437
0.0	1.3333	100	0.0150	2.4614	-7.0530	0.9900	9.5144	-57.3755	-24.3022	-1.1580	-1.1452
0.0	1.6667	125	0.0165	2.8817	-7.5738	0.9900	10.4555	-58.4170	-23.4615	-1.1561	-1.1425
0.0175	2.0	150	0.0077	2.7979	-9.1294	0.9900	11.9273	-61.5282	-23.6290	-1.1592	-1.1446
0.0	2.3333	175	0.0077	2.8004	-9.1559	0.9900	11.9563	-61.5813	-23.6240	-1.1592	-1.1445
0.0	2.6667	200	0.0098	3.5386	-9.1468	0.9900	12.6854	-61.5630	-22.1478	-1.1582	-1.1411
0.0	3.0	225	0.0098	3.5323	-9.1598	0.9900	12.6921	-61.5891	-22.1603	-1.1583	-1.1411
0.0	3.3333	250	0.0099	3.5384	-9.1504	0.9900	12.6888	-61.5704	-22.1482	-1.1580	-1.1408
0.0	3.6667	275	0.0101	3.5390	-9.1521	0.9900	12.6912	-61.5738	-22.1469	-1.1582	-1.1410
0.0173	4.0	300	0.0102	3.5300	-9.1689	0.9900	12.6988	-61.6072	-22.1650	-1.1582	-1.1410
0.0	4.3333	325	0.0095	3.5391	-9.1723	0.9900	12.7114	-61.6141	-22.1467	-1.1582	-1.1411
0.0173	4.6667	350	0.0098	3.5336	-9.1774	0.9900	12.7110	-61.6242	-22.1576	-1.1582	-1.1411
0.0	5.0	375	0.0100	3.5413	-9.1860	0.9900	12.7273	-61.6416	-22.1423	-1.1584	-1.1412
0.0173	5.3333	400	0.0097	3.5385	-9.1956	0.9900	12.7342	-61.6608	-22.1479	-1.1586	-1.1414
0.0173	5.6667	425	0.0099	3.5458	-9.1729	0.9900	12.7188	-61.6153	-22.1332	-1.1581	-1.1409
0.0	6.0	450	0.0095	3.5342	-9.2206	0.9900	12.7548	-61.7106	-22.1565	-1.1583	-1.1411
0.0	6.3333	475	0.0096	3.5378	-9.2207	0.9900	12.7585	-61.7109	-22.1492	-1.1585	-1.1413
0.0173	6.6667	500	0.0098	3.5344	-9.2288	0.9900	12.7632	-61.7271	-22.1561	-1.1588	-1.1415
0.0	7.0	525	0.0090	3.5387	-9.2492	0.9900	12.7878	-61.7678	-22.1475	-1.1587	-1.1414
0.0	7.3333	550	0.0092	3.5377	-9.2629	0.9900	12.8006	-61.7953	-22.1496	-1.1589	-1.1417
0.0173	7.6667	575	0.0093	3.5369	-9.2697	0.9900	12.8066	-61.8089	-22.1510	-1.1590	-1.1418
0.0	8.0	600	0.0094	3.5387	-9.2877	0.9900	12.8264	-61.8448	-22.1475	-1.1587	-1.1414
0.0347	8.3333	625	0.0098	3.5219	-9.2959	0.9900	12.8178	-61.8614	-22.1812	-1.1590	-1.1418
0.0	8.6667	650	0.0092	3.5332	-9.2917	0.9900	12.8249	-61.8529	-22.1584	-1.1589	-1.1416
0.0	9.0	675	0.0091	3.5324	-9.3041	0.9900	12.8365	-61.8776	-22.1600	-1.1591	-1.1418
0.0	9.3333	700	0.0096	3.5277	-9.3067	0.9900	12.8344	-61.8829	-22.1695	-1.1591	-1.1418
0.0	9.6667	725	0.0092	3.5429	-9.3040	0.9900	12.8470	-61.8776	-22.1390	-1.1591	-1.1418
0.0	10.0	750	0.0096	3.5350	-9.3114	0.9900	12.8464	-61.8923	-22.1549	-1.1588	-1.1415
0.0	10.3333	775	0.0094	3.5320	-9.3159	0.9900	12.8479	-61.9013	-22.1609	-1.1590	-1.1416
0.0	10.6667	800	0.0092	3.5430	-9.3106	0.9900	12.8535	-61.8906	-22.1389	-1.1591	-1.1418
0.0	11.0	825	0.0090	3.5293	-9.3094	0.9900	12.8387	-61.8883	-22.1663	-1.1589	-1.1416
0.0	11.3333	850	0.0093	3.5309	-9.3281	0.9900	12.8591	-61.9258	-22.1630	-1.1590	-1.1417
0.0173	11.6667	875	0.0093	3.5340	-9.3279	0.9900	12.8618	-61.9252	-22.1570	-1.1592	-1.1419
0.0	12.0	900	0.0092	3.5268	-9.3258	0.9900	12.8526	-61.9212	-22.1713	-1.1590	-1.1416
0.0	12.3333	925	0.0089	3.5337	-9.3216	0.9900	12.8553	-61.9127	-22.1576	-1.1590	-1.1417
0.0173	12.6667	950	0.0093	3.5404	-9.3113	0.9900	12.8518	-61.8922	-22.1440	-1.1591	-1.1419
0.0173	13.0	975	0.0094	3.5398	-9.3115	0.9900	12.8514	-61.8926	-22.1453	-1.1592	-1.1419
0.0	13.3333	1000	0.0094	3.5398	-9.3115	0.9900	12.8514	-61.8926	-22.1453	-1.1592	-1.1419

Framework versions

Transformers 4.41.2
Pytorch 2.0.0+cu117
Datasets 2.19.2
Tokenizers 0.19.1

Downloads last month: 6

Safetensors

Model size

8B params

Tensor type

F16

Model tree for tsavage68/UTI2_L3_1000steps_1e6rate_05beta_CSFTDPO

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

tsavage68/UTI_L3_1000steps_1e5rate_SFT

Finetuned

(39)

this model