Instructions to use tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO")
model = AutoModelForCausalLM.from_pretrained("tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO

SGLang

How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO with Docker Model Runner:
```
docker model run hf.co/tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO
```

Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO

This model is a fine-tuned version of tsavage68/Transaminitis_L3_1000rate_1e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.2656
Rewards/chosen: -5.8394
Rewards/rejected: -13.5464
Rewards/accuracies: 0.9500
Rewards/margins: 7.7070
Logps/rejected: -154.0191
Logps/chosen: -76.9285
Logits/rejected: -1.0971
Logits/chosen: -1.0952

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.8157	0.2	25	0.7130	-1.6812	-1.6422	0.2000	-0.0390	-34.9765	-35.3459	-1.0074	-1.0078
0.6531	0.4	50	0.5572	1.4920	1.0999	0.5400	0.3921	-7.5562	-3.6147	-0.6331	-0.6288
0.0069	0.6	75	0.0638	1.5026	-8.0172	0.9900	9.5198	-98.7265	-3.5080	-1.0032	-0.9076
1.4987	0.8	100	0.7768	-3.4746	-3.5322	0.5400	0.0576	-53.8765	-53.2803	-0.4138	-0.4136
0.7987	1.0	125	0.7220	-3.4829	-3.5110	0.5400	0.0281	-53.6649	-53.3632	-0.7087	-0.7087
0.7438	1.2	150	0.7114	-3.2843	-3.2535	0.4600	-0.0308	-51.0900	-51.3775	-1.0310	-1.0310
0.6949	1.4	175	0.7051	-3.3085	-3.2855	0.4000	-0.0230	-51.4100	-51.6195	-0.7593	-0.7593
0.7	1.6	200	0.7007	-3.3122	-3.2981	0.4400	-0.0141	-51.5352	-51.6561	-0.7261	-0.7261
0.7004	1.8	225	0.7092	-3.5268	-3.5014	0.4600	-0.0254	-53.5688	-53.8022	-1.0639	-1.0640
0.7056	2.0	250	0.7048	-3.3574	-3.3377	0.4800	-0.0197	-51.9312	-52.1080	-0.8329	-0.8329
0.6829	2.2	275	0.6964	-3.4182	-3.4152	0.5400	-0.0030	-52.7066	-52.7166	-1.0186	-1.0187
0.7101	2.4	300	0.6992	-4.3808	-4.3804	0.5400	-0.0003	-62.3591	-62.3421	-1.3638	-1.3638
0.7107	2.6	325	0.7081	-4.1483	-4.1266	0.4600	-0.0217	-59.8212	-60.0177	-1.3589	-1.3589
0.7035	2.8	350	0.6913	-3.0909	-3.0966	0.2900	0.0058	-49.5212	-49.4432	-0.7017	-0.7017
0.7112	3.0	375	0.7096	-4.4207	-4.3939	0.4600	-0.0268	-62.4938	-62.7416	-1.3752	-1.3752
0.659	3.2	400	0.7992	-4.2280	-4.1290	0.5200	-0.0990	-59.8449	-60.8146	-1.0809	-1.0815
0.6253	3.4	425	0.9164	-4.3837	-4.1124	0.5200	-0.2713	-59.6787	-62.3715	-0.7324	-0.7317
0.956	3.6	450	0.5266	-3.8419	-5.4570	0.6800	1.6151	-73.1246	-56.9532	-0.3747	-0.3742
0.5604	3.8	475	0.6506	-3.5933	-6.2168	0.7000	2.6234	-80.7223	-54.4675	-0.1960	-0.1952
0.8776	4.0	500	0.5657	-3.9281	-7.0564	0.8400	3.1284	-89.1191	-57.8147	-0.6674	-0.6680
0.4978	4.2	525	0.6285	-4.8602	-10.3518	0.8800	5.4916	-122.0728	-67.1361	-0.9244	-0.9236
1.0258	4.4	550	0.6966	-5.0528	-8.7895	0.8000	3.7367	-106.4495	-69.0625	-0.6216	-0.6205
0.3559	4.6	575	0.6527	-5.5366	-9.7092	0.8100	4.1726	-115.6466	-73.9002	-1.1615	-1.1603
0.2236	4.8	600	0.3743	-5.2783	-10.8881	0.9100	5.6099	-127.4360	-71.3169	-1.0731	-1.0714
0.0995	5.0	625	0.1816	-4.6140	-10.2504	0.9500	5.6364	-121.0588	-64.6745	-1.0550	-1.0504
0.4954	5.2	650	0.2771	-4.9474	-10.6256	0.9000	5.6781	-124.8103	-68.0087	-0.9020	-0.9007
0.2031	5.4	675	0.2731	-5.6955	-12.6949	0.9600	6.9994	-145.5037	-75.4888	-1.0406	-1.0388
0.3665	5.6	700	0.2912	-5.5615	-11.9434	0.9300	6.3819	-137.9883	-74.1489	-0.9311	-0.9288
0.132	5.8	725	0.2410	-6.2707	-13.3387	0.9400	7.0680	-151.9420	-81.2413	-1.0742	-1.0720
0.1044	6.0	750	0.2450	-6.0942	-13.2397	0.9500	7.1455	-150.9520	-79.4765	-1.0715	-1.0693
0.1984	6.2	775	0.2646	-6.1961	-13.4718	0.9500	7.2757	-153.2727	-80.4953	-1.0771	-1.0748
0.0156	6.4	800	0.3140	-6.1100	-13.6377	0.9500	7.5277	-154.9315	-79.6341	-1.1101	-1.1082
0.2682	6.6	825	0.2528	-5.9327	-13.5268	0.9600	7.5942	-153.8231	-77.8608	-1.0893	-1.0873
0.0011	6.8	850	0.2762	-5.9315	-13.5461	0.9500	7.6146	-154.0158	-77.8491	-1.0916	-1.0895
0.1031	7.0	875	0.2613	-5.8587	-13.5305	0.9500	7.6718	-153.8600	-77.1214	-1.0933	-1.0913
0.0034	7.2	900	0.2675	-5.8590	-13.5490	0.9500	7.6900	-154.0449	-77.1244	-1.0975	-1.0955
0.1314	7.4	925	0.2662	-5.8482	-13.5520	0.9500	7.7038	-154.0743	-77.0162	-1.0978	-1.0958
0.3318	7.6	950	0.2651	-5.8403	-13.5464	0.9500	7.7060	-154.0184	-76.9377	-1.0974	-1.0954
0.1093	7.8	975	0.2653	-5.8449	-13.5488	0.9500	7.7039	-154.0427	-76.9835	-1.0977	-1.0957
0.1808	8.0	1000	0.2656	-5.8394	-13.5464	0.9500	7.7070	-154.0191	-76.9285	-1.0971	-1.0952

Framework versions

Transformers 4.40.2
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

Downloads last month: 5

Safetensors

Model size

8B params

Tensor type

F16

Model tree for tsavage68/Transaminitis_L3_1000steps_1e5rate_01beta_CSFTDPO

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

tsavage68/Transaminitis_L3_1000rate_1e7_SFT

Finetuned

(17)

this model