Instructions to use tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT")
model = AutoModelForCausalLM.from_pretrained("tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT

SGLang

How to use tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT with Docker Model Runner:
```
docker model run hf.co/tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT
```

chat_1000_STEPS_03beta_1e6rate_CDPOSFT

This model is a fine-tuned version of tsavage68/chat_600STEPS_1e8rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.6755
Rewards/chosen: -0.5736
Rewards/rejected: -0.7849
Rewards/accuracies: 0.5121
Rewards/margins: 0.2113
Logps/rejected: -21.4183
Logps/chosen: -18.6666
Logits/rejected: -0.7004
Logits/chosen: -0.7002

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6903	0.0977	50	0.6898	0.0339	0.0260	0.4264	0.0078	-18.7152	-16.6418	-0.6000	-0.5999
0.6568	0.1953	100	0.6714	-0.1082	-0.1762	0.5099	0.0680	-19.3893	-17.1151	-0.6152	-0.6151
0.7127	0.2930	150	0.6820	-0.1152	-0.1845	0.4879	0.0693	-19.4168	-17.1385	-0.5988	-0.5986
0.7008	0.3906	200	0.6810	-0.1658	-0.2536	0.5055	0.0878	-19.6473	-17.3074	-0.5830	-0.5828
0.7256	0.4883	250	0.6858	-0.0964	-0.2054	0.4923	0.1090	-19.4867	-17.0761	-0.5766	-0.5764
0.6817	0.5859	300	0.6762	-0.2368	-0.3883	0.5187	0.1515	-20.0964	-17.5440	-0.6063	-0.6061
0.6486	0.6836	350	0.6850	-0.3387	-0.4688	0.5055	0.1301	-20.3646	-17.8836	-0.5899	-0.5897
0.651	0.7812	400	0.6734	-0.3143	-0.4779	0.5275	0.1636	-20.3950	-17.8025	-0.6197	-0.6195
0.6761	0.8789	450	0.6825	-0.1942	-0.3362	0.5011	0.1420	-19.9226	-17.4020	-0.5790	-0.5788
0.6615	0.9766	500	0.6798	-0.2233	-0.3810	0.4967	0.1578	-20.0720	-17.4988	-0.6050	-0.6048
0.3298	1.0742	550	0.6743	-0.2860	-0.4658	0.5055	0.1798	-20.3546	-17.7080	-0.6296	-0.6294
0.3296	1.1719	600	0.6753	-0.4100	-0.5995	0.5099	0.1894	-20.8002	-18.1215	-0.6547	-0.6545
0.3571	1.2695	650	0.6753	-0.4787	-0.6784	0.5143	0.1998	-21.0634	-18.3502	-0.6784	-0.6782
0.254	1.3672	700	0.6750	-0.5165	-0.7231	0.5099	0.2066	-21.2124	-18.4763	-0.6901	-0.6899
0.2391	1.4648	750	0.6754	-0.5562	-0.7657	0.5187	0.2095	-21.3543	-18.6087	-0.6964	-0.6962
0.3665	1.5625	800	0.6750	-0.5607	-0.7724	0.5055	0.2117	-21.3766	-18.6235	-0.6992	-0.6990
0.315	1.6602	850	0.6758	-0.5717	-0.7824	0.5077	0.2106	-21.4099	-18.6604	-0.7006	-0.7004
0.3595	1.7578	900	0.6761	-0.5738	-0.7840	0.5077	0.2101	-21.4152	-18.6674	-0.7007	-0.7005
0.3196	1.8555	950	0.6747	-0.5736	-0.7866	0.5077	0.2130	-21.4241	-18.6667	-0.7012	-0.7010
0.2841	1.9531	1000	0.6755	-0.5736	-0.7849	0.5121	0.2113	-21.4183	-18.6666	-0.7004	-0.7002

Framework versions

Transformers 4.40.1
Pytorch 2.0.0+cu117
Datasets 2.19.0
Tokenizers 0.19.1

Downloads last month: 5

Safetensors

Model size

7B params

Tensor type

F16

Model tree for tsavage68/chat_1000_STEPS_03beta_1e6rate_CDPOSFT

Base model

meta-llama/Llama-2-7b-chat-hf

Finetuned

tsavage68/chat_600STEPS_1e8rate_SFT

Finetuned

(23)

this model