Instructions to use tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO")
model = AutoModelForCausalLM.from_pretrained("tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO

SGLang

How to use tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO with Docker Model Runner:
```
docker model run hf.co/tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO
```

UTI_L3_1000steps_1e7rate_01beta_CSFTDPO

This model is a fine-tuned version of tsavage68/UTI_L3_1000steps_1e5rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1442
Rewards/chosen: 0.3972
Rewards/rejected: -2.1672
Rewards/accuracies: 0.9900
Rewards/margins: 2.5644
Logps/rejected: -84.8662
Logps/chosen: -28.5068
Logits/rejected: -1.3311
Logits/chosen: -1.3133

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6922	0.3333	25	0.6925	-0.0003	-0.0016	0.5400	0.0013	-63.2107	-32.4819	-1.3229	-1.3078
0.6843	0.6667	50	0.6806	0.0044	-0.0210	0.8100	0.0254	-63.4048	-32.4353	-1.3232	-1.3080
0.6624	1.0	75	0.6486	0.0165	-0.0759	0.9400	0.0923	-63.9532	-32.3144	-1.3233	-1.3082
0.5995	1.3333	100	0.5895	0.0366	-0.1897	0.9700	0.2262	-65.0915	-32.1134	-1.3238	-1.3089
0.5297	1.6667	125	0.5190	0.0681	-0.3413	0.9900	0.4094	-66.6078	-31.7979	-1.3241	-1.3092
0.4684	2.0	150	0.4586	0.0995	-0.4924	0.9900	0.5919	-68.1188	-31.4842	-1.3242	-1.3094
0.3906	2.3333	175	0.4069	0.1251	-0.6562	0.9900	0.7813	-69.7570	-31.2282	-1.3245	-1.3096
0.3638	2.6667	200	0.3679	0.1521	-0.7841	0.9900	0.9363	-71.0359	-30.9576	-1.3251	-1.3102
0.3253	3.0	225	0.3306	0.1774	-0.9332	0.9900	1.1106	-72.5262	-30.7046	-1.3254	-1.3105
0.3299	3.3333	250	0.3023	0.1996	-1.0585	0.9900	1.2581	-73.7797	-30.4830	-1.3260	-1.3110
0.2537	3.6667	275	0.2752	0.2237	-1.1989	0.9900	1.4225	-75.1834	-30.2425	-1.3261	-1.3109
0.2526	4.0	300	0.2526	0.2452	-1.3215	0.9900	1.5667	-76.4094	-30.0269	-1.3268	-1.3114
0.1951	4.3333	325	0.2341	0.2656	-1.4371	0.9900	1.7028	-77.5660	-29.8226	-1.3271	-1.3115
0.2002	4.6667	350	0.2197	0.2836	-1.5313	0.9900	1.8149	-78.5079	-29.6433	-1.3279	-1.3121
0.212	5.0	375	0.2053	0.3045	-1.6219	0.9900	1.9264	-79.4134	-29.4336	-1.3282	-1.3121
0.1959	5.3333	400	0.1940	0.3221	-1.7035	0.9900	2.0257	-80.2299	-29.2576	-1.3288	-1.3124
0.1676	5.6667	425	0.1846	0.3350	-1.7864	0.9900	2.1214	-81.0588	-29.1291	-1.3292	-1.3126
0.1475	6.0	450	0.1769	0.3445	-1.8487	0.9900	2.1932	-81.6814	-29.0337	-1.3295	-1.3128
0.1344	6.3333	475	0.1710	0.3549	-1.9032	0.9900	2.2581	-82.2267	-28.9301	-1.3298	-1.3129
0.1697	6.6667	500	0.1652	0.3627	-1.9552	0.9900	2.3178	-82.7465	-28.8523	-1.3300	-1.3129
0.1423	7.0	525	0.1605	0.3708	-1.9950	0.9900	2.3658	-83.1446	-28.7710	-1.3303	-1.3131
0.1229	7.3333	550	0.1569	0.3783	-2.0319	0.9900	2.4102	-83.5133	-28.6961	-1.3305	-1.3131
0.1507	7.6667	575	0.1537	0.3823	-2.0654	0.9900	2.4476	-83.8482	-28.6561	-1.3307	-1.3132
0.1373	8.0	600	0.1512	0.3851	-2.0959	0.9900	2.4810	-84.1538	-28.6278	-1.3309	-1.3133
0.1324	8.3333	625	0.1497	0.3897	-2.1128	0.9900	2.5026	-84.3230	-28.5817	-1.3310	-1.3133
0.1095	8.6667	650	0.1476	0.3906	-2.1327	0.9900	2.5233	-84.5217	-28.5733	-1.3309	-1.3132
0.1282	9.0	675	0.1465	0.3929	-2.1449	0.9900	2.5378	-84.6436	-28.5502	-1.3310	-1.3133
0.1155	9.3333	700	0.1458	0.3943	-2.1507	0.9900	2.5450	-84.7017	-28.5359	-1.3311	-1.3134
0.1118	9.6667	725	0.1449	0.3958	-2.1591	0.9900	2.5549	-84.7855	-28.5210	-1.3312	-1.3134
0.1124	10.0	750	0.1451	0.3968	-2.1625	0.9900	2.5593	-84.8200	-28.5114	-1.3311	-1.3133
0.0737	10.3333	775	0.1445	0.3972	-2.1645	0.9900	2.5617	-84.8398	-28.5074	-1.3314	-1.3136
0.1207	10.6667	800	0.1444	0.3965	-2.1644	0.9900	2.5609	-84.8389	-28.5145	-1.3312	-1.3133
0.1324	11.0	825	0.1442	0.3985	-2.1680	0.9900	2.5665	-84.8743	-28.4940	-1.3311	-1.3132
0.103	11.3333	850	0.1444	0.3963	-2.1663	0.9900	2.5626	-84.8578	-28.5159	-1.3312	-1.3134
0.1459	11.6667	875	0.1439	0.3974	-2.1681	0.9900	2.5654	-84.8755	-28.5054	-1.3314	-1.3135
0.1244	12.0	900	0.1442	0.3980	-2.1683	0.9900	2.5663	-84.8780	-28.4994	-1.3313	-1.3135
0.1208	12.3333	925	0.1442	0.3973	-2.1670	0.9900	2.5642	-84.8642	-28.5061	-1.3311	-1.3133
0.1209	12.6667	950	0.1442	0.3972	-2.1672	0.9900	2.5644	-84.8662	-28.5068	-1.3311	-1.3133
0.1852	13.0	975	0.1442	0.3972	-2.1672	0.9900	2.5644	-84.8662	-28.5068	-1.3311	-1.3133
0.1236	13.3333	1000	0.1442	0.3972	-2.1672	0.9900	2.5644	-84.8662	-28.5068	-1.3311	-1.3133

Framework versions

Transformers 4.41.2
Pytorch 2.0.0+cu117
Datasets 2.19.2
Tokenizers 0.19.1

Downloads last month: 6

Safetensors

Model size

8B params

Tensor type

F16

Model tree for tsavage68/UTI_L3_1000steps_1e7rate_01beta_CSFTDPO

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

tsavage68/UTI_L3_1000steps_1e5rate_SFT

Finetuned

(39)

this model