Instructions to use tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO")
model = AutoModelForCausalLM.from_pretrained("tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO

SGLang

How to use tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO with Docker Model Runner:
```
docker model run hf.co/tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO
```

Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO

This model is a fine-tuned version of tsavage68/Transaminitis_L3_1000rate_1e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1430
Rewards/chosen: 2.5912
Rewards/rejected: -3.0443
Rewards/accuracies: 0.9200
Rewards/margins: 5.6356
Logps/rejected: -24.6434
Logps/chosen: -13.3518
Logits/rejected: -1.0678
Logits/chosen: -1.0599

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6857	0.2	25	0.6895	-0.0206	-0.0288	0.5100	0.0082	-18.6123	-18.5753	-1.0653	-1.0641
0.6912	0.4	50	0.6888	-0.1407	-0.1512	0.5300	0.0106	-18.8572	-18.8156	-1.0675	-1.0663
0.6956	0.6	75	0.6978	-0.1002	-0.1011	0.4600	0.0008	-18.7568	-18.7347	-1.0682	-1.0669
0.6647	0.8	100	0.7297	-0.2211	-0.2283	0.4600	0.0071	-19.0112	-18.9765	-1.0701	-1.0690
0.7128	1.0	125	0.6805	-0.6632	-0.8200	0.5400	0.1568	-20.1946	-19.8606	-1.0730	-1.0717
0.6584	1.2	150	0.6804	0.2045	0.1201	0.4600	0.0843	-18.3144	-18.1253	-1.0688	-1.0675
0.6173	1.4	175	0.6162	-0.0012	-0.1827	0.7400	0.1816	-18.9201	-18.5365	-1.0737	-1.0722
0.5735	1.6	200	0.5756	0.0705	-0.2240	0.7700	0.2944	-19.0026	-18.3933	-1.0752	-1.0739
0.5544	1.8	225	0.5409	0.5448	0.1622	0.7700	0.3826	-18.2303	-17.4446	-1.0753	-1.0739
0.4913	2.0	250	0.4928	0.8241	0.3088	0.8500	0.5153	-17.9371	-16.8860	-1.0761	-1.0744
0.4418	2.2	275	0.4281	1.0103	0.2873	0.8600	0.7230	-17.9801	-16.5137	-1.0761	-1.0743
0.3932	2.4	300	0.3699	1.1591	0.2051	0.8900	0.9540	-18.1444	-16.2160	-1.0770	-1.0748
0.3476	2.6	325	0.3822	1.8158	0.6290	0.8500	1.1869	-17.2967	-14.9025	-1.0747	-1.0724
0.1934	2.8	350	0.2438	1.8838	0.0549	0.9300	1.8289	-18.4449	-14.7666	-1.0768	-1.0734
0.2502	3.0	375	0.2149	1.9679	-0.1145	0.9100	2.0824	-18.7836	-14.5983	-1.0751	-1.0716
0.2345	3.2	400	0.1910	2.2017	-0.2910	0.9300	2.4927	-19.1366	-14.1307	-1.0746	-1.0705
0.1229	3.4	425	0.1688	2.4784	-0.4633	0.9300	2.9416	-19.4812	-13.5775	-1.0736	-1.0687
0.1974	3.6	450	0.1562	2.3632	-0.9671	0.9300	3.3303	-20.4888	-13.8078	-1.0738	-1.0685
0.1073	3.8	475	0.1528	2.4826	-1.1353	0.9200	3.6179	-20.8253	-13.5691	-1.0736	-1.0680
0.1973	4.0	500	0.1911	2.6293	-1.1284	0.9400	3.7577	-20.8116	-13.2756	-1.0711	-1.0654
0.011	4.2	525	0.1344	2.5816	-1.7865	0.9300	4.3681	-22.1277	-13.3710	-1.0717	-1.0650
0.1103	4.4	550	0.1405	2.5994	-2.0187	0.9300	4.6181	-22.5922	-13.3355	-1.0715	-1.0647
0.0374	4.6	575	0.1405	2.5994	-2.3029	0.9300	4.9023	-23.1606	-13.3355	-1.0705	-1.0635
0.1784	4.8	600	0.1593	2.5902	-2.3960	0.9400	4.9862	-23.3466	-13.3538	-1.0695	-1.0624
0.0403	5.0	625	0.1408	2.5556	-2.6439	0.9400	5.1995	-23.8425	-13.4230	-1.0700	-1.0625
0.1204	5.2	650	0.1479	2.5770	-2.7623	0.9400	5.3393	-24.0793	-13.3802	-1.0692	-1.0618
0.1457	5.4	675	0.1417	2.6333	-2.7844	0.9300	5.4177	-24.1235	-13.2676	-1.0686	-1.0609
0.0215	5.6	700	0.1538	2.6044	-2.8717	0.9300	5.4762	-24.2982	-13.3253	-1.0688	-1.0610
0.037	5.8	725	0.1452	2.5901	-2.9650	0.9300	5.5551	-24.4847	-13.3540	-1.0694	-1.0616
0.2717	6.0	750	0.1378	2.6100	-3.0205	0.9400	5.6305	-24.5957	-13.3143	-1.0675	-1.0596
0.1493	6.2	775	0.1401	2.5893	-3.0192	0.9300	5.6085	-24.5931	-13.3556	-1.0695	-1.0616
0.233	6.4	800	0.1367	2.5602	-3.0801	0.9400	5.6403	-24.7149	-13.4137	-1.0691	-1.0611
0.0528	6.6	825	0.1422	2.5944	-3.0372	0.9400	5.6317	-24.6291	-13.3453	-1.0690	-1.0611
0.0361	6.8	850	0.1325	2.5922	-3.0459	0.9500	5.6381	-24.6465	-13.3498	-1.0680	-1.0600
0.0325	7.0	875	0.1407	2.5860	-3.0411	0.9200	5.6271	-24.6368	-13.3621	-1.0678	-1.0600
0.0257	7.2	900	0.1408	2.5790	-3.0416	0.9200	5.6206	-24.6378	-13.3762	-1.0677	-1.0599
0.2493	7.4	925	0.1432	2.5875	-3.0479	0.9200	5.6354	-24.6505	-13.3592	-1.0679	-1.0600
0.2282	7.6	950	0.1430	2.5912	-3.0443	0.9200	5.6356	-24.6434	-13.3518	-1.0678	-1.0599
0.1122	7.8	975	0.1430	2.5912	-3.0443	0.9200	5.6356	-24.6434	-13.3518	-1.0678	-1.0599
0.1086	8.0	1000	0.1430	2.5912	-3.0443	0.9200	5.6356	-24.6434	-13.3518	-1.0678	-1.0599

Framework versions

Transformers 4.40.2
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

Downloads last month: 8

Safetensors

Model size

8B params

Tensor type

F16

Model tree for tsavage68/Transaminitis_L3_1000steps_1e7rate_05beta_CSFTDPO

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

tsavage68/Transaminitis_L3_1000rate_1e7_SFT

Finetuned

(17)

this model