Instructions to use tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO")
model = AutoModelForCausalLM.from_pretrained("tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO

SGLang

How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO with Docker Model Runner:
```
docker model run hf.co/tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO
```

Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO

This model is a fine-tuned version of tsavage68/Transaminitis_L3_1000rate_1e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5385
Rewards/chosen: -13.1985
Rewards/rejected: -15.4515
Rewards/accuracies: 0.7800
Rewards/margins: 2.2530
Logps/rejected: -70.0598
Logps/chosen: -62.5292
Logits/rejected: -0.0142
Logits/chosen: -0.0142

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
1.3908	0.2	25	1.2815	-0.1831	-0.0153	0.4600	-0.1678	-18.6058	-19.1447	-0.7380	-0.7444
0.7696	0.4	50	0.6447	-2.0796	-3.1294	0.4800	1.0498	-28.9860	-25.4662	-0.6995	-0.6959
1.4448	0.6	75	1.1250	2.3681	1.6567	0.5400	0.7115	-13.0325	-10.6404	-1.2011	-1.1986
5.4772	0.8	100	0.9871	-6.9709	-6.8051	0.4600	-0.1658	-41.2385	-41.7705	0.9489	0.9489
2.3001	1.0	125	0.7456	-8.7651	-8.7572	0.5400	-0.0079	-47.7455	-47.7513	-0.1259	-0.1259
0.7493	1.2	150	0.8732	-9.5881	-9.4530	0.4600	-0.1351	-50.0647	-50.4944	0.5943	0.5943
0.8138	1.4	175	0.7245	-9.0738	-9.0405	0.5400	-0.0334	-48.6896	-48.7803	0.6213	0.6212
0.8059	1.6	200	0.7545	-9.4290	-9.3426	0.4600	-0.0864	-49.6966	-49.9642	0.5425	0.5425
1.1375	1.8	225	0.8646	-9.5637	-9.4326	0.4600	-0.1310	-49.9968	-50.4131	0.4615	0.4615
0.8527	2.0	250	0.7264	-9.3869	-9.3241	0.2700	-0.0627	-49.6351	-49.8238	0.4676	0.4675
0.7399	2.2	275	0.7555	-11.2127	-11.2222	0.5400	0.0095	-55.9621	-55.9100	-0.0369	-0.0369
0.8543	2.4	300	0.7201	-9.4861	-9.4619	0.5400	-0.0242	-50.0944	-50.1545	0.2903	0.2903
0.7482	2.6	325	0.7498	-9.6111	-9.5315	0.4600	-0.0797	-50.3262	-50.5713	0.2174	0.2174
0.7855	2.8	350	0.7175	-9.2874	-9.2716	0.5400	-0.0158	-49.4600	-49.4921	0.2093	0.2093
0.8186	3.0	375	0.7371	-15.1108	-15.1001	0.5400	-0.0107	-68.8884	-68.9036	0.2163	0.2165
2.0277	3.2	400	0.7537	-9.4310	-9.3464	0.4600	-0.0846	-49.7093	-49.9710	0.1700	0.1700
0.7813	3.4	425	0.7032	-9.2991	-9.3052	0.5300	0.0061	-49.5722	-49.5312	0.1934	0.1934
0.7558	3.6	450	0.7507	-9.2119	-9.1319	0.4600	-0.0800	-48.9944	-49.2406	0.1106	0.1106
0.7805	3.8	475	0.7580	-8.8615	-8.8256	0.5400	-0.0359	-47.9734	-48.0724	-0.1360	-0.1360
0.7676	4.0	500	0.4827	-11.2587	-13.0016	0.7600	1.7429	-61.8933	-56.0631	0.1036	0.1034
0.7259	4.2	525	0.5377	-14.1470	-17.2492	0.7800	3.1022	-76.0520	-65.6908	0.0223	0.0221
0.5792	4.4	550	4.9230	-23.8146	-27.4972	0.6700	3.6825	-110.2119	-97.9164	-0.2570	-0.2572
0.7514	4.6	575	0.7208	-15.0032	-16.9616	0.7600	1.9584	-75.0933	-68.5450	0.2455	0.2448
0.3657	4.8	600	0.5002	-11.0206	-13.3204	0.7900	2.2998	-62.9559	-55.2696	0.0339	0.0339
0.3482	5.0	625	0.4679	-11.7643	-14.2910	0.7700	2.5267	-66.1913	-57.7484	0.0154	0.0154
0.692	5.2	650	0.4165	-12.0861	-13.4630	0.8200	1.3769	-63.4313	-58.8213	0.0440	0.0440
0.7492	5.4	675	0.5138	-11.3952	-13.1124	0.6700	1.7172	-62.2628	-56.5183	-0.0121	-0.0121
0.4567	5.6	700	0.5206	-12.6196	-15.1706	0.8000	2.5510	-69.1234	-60.5996	0.0016	0.0016
0.5014	5.8	725	0.5012	-13.2139	-16.0835	0.8100	2.8695	-72.1662	-62.5807	0.0120	0.0120
0.4501	6.0	750	0.4553	-12.9387	-15.7396	0.8100	2.8009	-71.0200	-61.6633	0.0061	0.0061
0.4102	6.2	775	0.4980	-12.8765	-15.8000	0.8100	2.9235	-71.2213	-61.4559	-0.0279	-0.0280
0.3204	6.4	800	0.4780	-12.8167	-15.2524	0.8200	2.4357	-69.3959	-61.2566	-0.0151	-0.0151
0.356	6.6	825	0.6408	-13.5884	-15.8776	0.7700	2.2893	-71.4801	-63.8287	-0.0146	-0.0146
0.6142	6.8	850	0.6131	-13.4279	-15.5537	0.7600	2.1258	-70.4003	-63.2937	-0.0129	-0.0130
0.3346	7.0	875	0.5203	-13.0899	-15.3438	0.7900	2.2539	-69.7008	-62.1672	-0.0134	-0.0135
0.5631	7.2	900	0.5492	-13.2280	-15.4542	0.7800	2.2262	-70.0687	-62.6276	-0.0134	-0.0135
0.2912	7.4	925	0.5433	-13.2100	-15.4562	0.7800	2.2462	-70.0753	-62.5676	-0.0139	-0.0140
0.2822	7.6	950	0.5404	-13.2017	-15.4485	0.7800	2.2468	-70.0496	-62.5398	-0.0138	-0.0139
0.3688	7.8	975	0.5385	-13.2010	-15.4516	0.7800	2.2506	-70.0602	-62.5377	-0.0138	-0.0139
0.3395	8.0	1000	0.5385	-13.1985	-15.4515	0.7800	2.2530	-70.0598	-62.5292	-0.0142	-0.0142

Framework versions

Transformers 4.40.2
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

Downloads last month: 7

Safetensors

Model size

8B params

Tensor type

F16

Model tree for tsavage68/Transaminitis_L3_1000steps_1e5rate_03beta_CSFTDPO

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

tsavage68/Transaminitis_L3_1000rate_1e7_SFT

Finetuned

(17)

this model