Instructions to use tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO")
model = AutoModelForCausalLM.from_pretrained("tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO

SGLang

How to use tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO with Docker Model Runner:
```
docker model run hf.co/tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO
```

UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO

This model is a fine-tuned version of tsavage68/UTI_L3_1000steps_1e5rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.2514
Rewards/chosen: 1.0384
Rewards/rejected: -2.5887
Rewards/accuracies: 0.6500
Rewards/margins: 3.6271
Logps/rejected: -37.1137
Logps/chosen: -15.6419
Logits/rejected: -1.1635
Logits/chosen: -1.1553

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6944	0.3333	25	0.6884	0.0045	-0.0054	0.3900	0.0099	-28.5027	-19.0882	-1.1523	-1.1488
0.6816	0.6667	50	0.6751	0.0295	-0.0076	0.5600	0.0370	-28.5099	-19.0051	-1.1524	-1.1488
0.6338	1.0	75	0.6279	0.0961	-0.0432	0.6300	0.1392	-28.6285	-18.7831	-1.1525	-1.1488
0.5045	1.3333	100	0.5423	0.2255	-0.1296	0.6400	0.3551	-28.9166	-18.3516	-1.1531	-1.1490
0.418	1.6667	125	0.4514	0.3749	-0.2772	0.6500	0.6522	-29.4088	-17.8536	-1.1538	-1.1494
0.4462	2.0	150	0.3918	0.4758	-0.4504	0.6500	0.9261	-29.9859	-17.5174	-1.1550	-1.1502
0.3407	2.3333	175	0.3498	0.5690	-0.6230	0.6500	1.1920	-30.5612	-17.2065	-1.1554	-1.1504
0.2843	2.6667	200	0.3196	0.6474	-0.8203	0.6500	1.4677	-31.2189	-16.9454	-1.1560	-1.1507
0.2531	3.0	225	0.2983	0.7087	-1.0237	0.6500	1.7323	-31.8969	-16.7411	-1.1565	-1.1508
0.2528	3.3333	250	0.2847	0.7720	-1.2036	0.6500	1.9756	-32.4966	-16.5300	-1.1572	-1.1512
0.1933	3.6667	275	0.2751	0.8130	-1.3994	0.6500	2.2124	-33.1494	-16.3934	-1.1582	-1.1519
0.2881	4.0	300	0.2685	0.8570	-1.5612	0.6500	2.4182	-33.6888	-16.2466	-1.1590	-1.1525
0.2511	4.3333	325	0.2642	0.8844	-1.7227	0.6500	2.6071	-34.2270	-16.1553	-1.1598	-1.1530
0.1639	4.6667	350	0.2612	0.9186	-1.8702	0.6500	2.7887	-34.7186	-16.0414	-1.1606	-1.1536
0.2312	5.0	375	0.2579	0.9429	-2.0206	0.6500	2.9635	-35.2199	-15.9604	-1.1613	-1.1540
0.2301	5.3333	400	0.2558	0.9559	-2.1492	0.6500	3.1051	-35.6486	-15.9168	-1.1616	-1.1542
0.2289	5.6667	425	0.2549	0.9690	-2.2386	0.6500	3.2076	-35.9468	-15.8733	-1.1621	-1.1545
0.2106	6.0	450	0.2540	0.9873	-2.3044	0.6500	3.2918	-36.1660	-15.8122	-1.1622	-1.1545
0.2794	6.3333	475	0.2538	1.0007	-2.3501	0.6500	3.3509	-36.3185	-15.7675	-1.1626	-1.1548
0.3133	6.6667	500	0.2535	0.9979	-2.3982	0.6500	3.3960	-36.4786	-15.7771	-1.1627	-1.1548
0.2103	7.0	525	0.2528	1.0083	-2.4563	0.6500	3.4646	-36.6723	-15.7423	-1.1632	-1.1551
0.177	7.3333	550	0.2524	1.0160	-2.4923	0.6500	3.5083	-36.7923	-15.7167	-1.1630	-1.1549
0.2615	7.6667	575	0.2523	1.0217	-2.5108	0.6500	3.5325	-36.8539	-15.6977	-1.1631	-1.1550
0.2613	8.0	600	0.2522	1.0242	-2.5323	0.6500	3.5566	-36.9258	-15.6891	-1.1630	-1.1548
0.2269	8.3333	625	0.2518	1.0294	-2.5471	0.6500	3.5765	-36.9751	-15.6721	-1.1635	-1.1553
0.1059	8.6667	650	0.2521	1.0271	-2.5645	0.6500	3.5915	-37.0329	-15.6797	-1.1636	-1.1553
0.2274	9.0	675	0.2516	1.0319	-2.5726	0.6500	3.6045	-37.0601	-15.6638	-1.1633	-1.1551
0.2265	9.3333	700	0.2517	1.0299	-2.5852	0.6500	3.6151	-37.1020	-15.6702	-1.1635	-1.1553
0.2266	9.6667	725	0.2517	1.0257	-2.5906	0.6500	3.6162	-37.1199	-15.6844	-1.1635	-1.1552
0.3151	10.0	750	0.2517	1.0347	-2.5867	0.6500	3.6214	-37.1071	-15.6543	-1.1633	-1.1551
0.2092	10.3333	775	0.2517	1.0320	-2.5944	0.6500	3.6264	-37.1328	-15.6633	-1.1636	-1.1553
0.2267	10.6667	800	0.2513	1.0270	-2.5989	0.6500	3.6258	-37.1475	-15.6801	-1.1633	-1.1550
0.2444	11.0	825	0.2514	1.0330	-2.5924	0.6500	3.6254	-37.1261	-15.6599	-1.1635	-1.1553
0.2957	11.3333	850	0.2516	1.0340	-2.5935	0.6500	3.6275	-37.1297	-15.6566	-1.1635	-1.1552
0.1754	11.6667	875	0.2516	1.0332	-2.5913	0.6500	3.6245	-37.1224	-15.6594	-1.1634	-1.1551
0.1594	12.0	900	0.2515	1.0326	-2.5942	0.6500	3.6268	-37.1320	-15.6612	-1.1635	-1.1552
0.1921	12.3333	925	0.2514	1.0378	-2.5889	0.6500	3.6266	-37.1143	-15.6441	-1.1635	-1.1552
0.2439	12.6667	950	0.2514	1.0384	-2.5887	0.6500	3.6271	-37.1137	-15.6419	-1.1635	-1.1553
0.2099	13.0	975	0.2514	1.0384	-2.5887	0.6500	3.6271	-37.1137	-15.6419	-1.1635	-1.1553
0.313	13.3333	1000	0.2514	1.0384	-2.5887	0.6500	3.6271	-37.1137	-15.6419	-1.1635	-1.1553

Framework versions

Transformers 4.41.2
Pytorch 2.0.0+cu117
Datasets 2.19.2
Tokenizers 0.19.1

Downloads last month: 6

Safetensors

Model size

8B params

Tensor type

F16

Model tree for tsavage68/UTI2_L3_1000steps_1e7rate_03beta_CSFTDPO

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

tsavage68/UTI_L3_1000steps_1e5rate_SFT

Finetuned

(39)

this model