Instructions to use tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT")
model = AutoModelForCausalLM.from_pretrained("tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT

SGLang

How to use tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT with Docker Model Runner:
```
docker model run hf.co/tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT
```

chat_1000_STEPS_03beta_CDPOSFT

This model is a fine-tuned version of tsavage68/chat_600STEPS_1e8rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.4273
Rewards/chosen: -2.7830
Rewards/rejected: -3.1629
Rewards/accuracies: 0.4571
Rewards/margins: 0.3799
Logps/rejected: -29.3449
Logps/chosen: -26.0311
Logits/rejected: 0.7969
Logits/chosen: 0.7973

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.8463	0.0977	50	0.7942	-1.4469	-1.5636	0.4703	0.1167	-24.0138	-21.5775	-0.4872	-0.4869
1.0828	0.1953	100	1.2743	-1.3396	-1.1843	0.3714	-0.1553	-22.7497	-21.2200	0.0456	0.0465
1.8211	0.2930	150	1.4868	-4.4773	-4.4154	0.3846	-0.0618	-33.5200	-31.6789	-0.0645	-0.0641
1.5571	0.3906	200	1.3347	-1.3781	-1.2581	0.3714	-0.1200	-22.9957	-21.3483	0.1053	0.1057
1.4698	0.4883	250	1.4544	-2.2266	-2.0161	0.3692	-0.2105	-25.5224	-24.1767	0.6310	0.6312
1.3848	0.5859	300	1.4989	-1.5043	-1.2708	0.3407	-0.2335	-23.0381	-21.7690	1.0175	1.0178
1.608	0.6836	350	1.5122	-1.8704	-1.6039	0.3626	-0.2666	-24.1482	-22.9894	0.2808	0.2809
1.3065	0.7812	400	1.3363	-2.6927	-2.7556	0.4044	0.0630	-27.9874	-25.7301	-0.0631	-0.0627
1.3358	0.8789	450	1.4411	-3.3344	-3.3172	0.4132	-0.0172	-29.8592	-27.8692	0.8838	0.8844
1.4118	0.9766	500	1.4283	-3.1040	-3.0583	0.3978	-0.0457	-28.9962	-27.1012	1.4387	1.4389
0.3497	1.0742	550	1.4828	-2.9569	-3.0649	0.4220	0.1081	-29.0184	-26.6108	1.1567	1.1568
0.3298	1.1719	600	1.5122	-2.8698	-3.2356	0.4396	0.3658	-29.5874	-26.3207	0.9066	0.9067
0.339	1.2695	650	1.5409	-3.8192	-4.1085	0.4308	0.2893	-32.4970	-29.4854	0.7328	0.7330
0.6687	1.3672	700	1.4092	-2.7766	-3.0696	0.4352	0.2930	-29.0340	-26.0099	0.8641	0.8644
0.2321	1.4648	750	1.4316	-2.7067	-3.0246	0.4484	0.3179	-28.8840	-25.7770	0.8522	0.8525
0.5027	1.5625	800	1.4235	-2.7303	-3.0988	0.4549	0.3685	-29.1314	-25.8558	0.8129	0.8132
0.469	1.6602	850	1.4289	-2.7971	-3.1797	0.4549	0.3825	-29.4008	-26.0784	0.7913	0.7917
0.4367	1.7578	900	1.4267	-2.7870	-3.1689	0.4593	0.3819	-29.3650	-26.0447	0.7954	0.7957
0.3005	1.8555	950	1.4262	-2.7846	-3.1662	0.4593	0.3816	-29.3560	-26.0368	0.7965	0.7968
0.2266	1.9531	1000	1.4273	-2.7830	-3.1629	0.4571	0.3799	-29.3449	-26.0311	0.7969	0.7973

Framework versions

Transformers 4.40.1
Pytorch 2.0.0+cu117
Datasets 2.19.0
Tokenizers 0.19.1

Downloads last month: 6

Safetensors

Model size

7B params

Tensor type

F16

Model tree for tsavage68/chat_1000_STEPS_03beta_1e5rate_CDPOSFT

Base model

meta-llama/Llama-2-7b-chat-hf

Finetuned

tsavage68/chat_600STEPS_1e8rate_SFT

Finetuned

(23)

this model