Instructions to use tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT")
model = AutoModelForCausalLM.from_pretrained("tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT

SGLang

How to use tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT with Docker Model Runner:
```
docker model run hf.co/tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT
```

Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT

This model is a fine-tuned version of tsavage68/mistralit2_1000_STEPS_5e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.9166
Rewards/chosen: -3.7686
Rewards/rejected: -6.1250
Rewards/accuracies: 0.6659
Rewards/margins: 2.3564
Logps/rejected: -87.8066
Logps/chosen: -61.3577
Logits/rejected: -1.6465
Logits/chosen: -1.6475

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5681	0.0977	50	0.5897	-0.2848	-0.7807	0.6220	0.4959	-34.3643	-26.5198	-2.0822	-2.0819
0.5454	0.1953	100	0.7078	-1.8828	-2.8036	0.6198	0.9208	-54.5930	-42.5001	-1.8193	-1.8193
0.7509	0.2930	150	0.6446	-0.7755	-1.2878	0.6022	0.5123	-39.4349	-31.4272	-2.3220	-2.3221
0.5784	0.3906	200	0.6519	-1.2019	-1.8232	0.6132	0.6213	-44.7889	-35.6908	-2.2690	-2.2690
0.7042	0.4883	250	0.6671	0.2266	-0.1145	0.5560	0.3411	-27.7019	-21.4055	-2.6333	-2.6332
0.5541	0.5859	300	0.6786	-1.5906	-2.3448	0.5868	0.7542	-50.0051	-39.5776	-2.7458	-2.7459
0.5497	0.6836	350	0.7057	-1.7745	-2.5281	0.6154	0.7537	-51.8382	-41.4164	-2.3898	-2.3902
0.7584	0.7812	400	0.6186	-0.5869	-1.5391	0.6637	0.9522	-41.9478	-29.5405	-2.4709	-2.4713
0.5936	0.8789	450	0.6546	-1.3995	-2.1895	0.6176	0.7901	-48.4523	-37.6665	-2.4013	-2.4017
0.4586	0.9766	500	0.6362	-1.1700	-2.0896	0.6330	0.9196	-47.4530	-35.3713	-2.0695	-2.0701
0.1066	1.0742	550	0.8659	-3.6103	-5.5577	0.6593	1.9474	-82.1337	-59.7750	-1.2953	-1.2968
0.1745	1.1719	600	0.8617	-3.4521	-5.5739	0.6374	2.1218	-82.2959	-58.1929	-1.4289	-1.4301
0.2043	1.2695	650	0.8388	-3.1557	-5.3780	0.6615	2.2223	-80.3369	-55.2288	-1.5113	-1.5124
0.391	1.3672	700	0.9441	-3.8602	-6.1157	0.6659	2.2555	-87.7136	-62.2736	-1.6592	-1.6602
0.0658	1.4648	750	0.9386	-3.7565	-6.1054	0.6549	2.3489	-87.6110	-61.2367	-1.6260	-1.6270
0.1568	1.5625	800	0.9240	-3.7963	-6.1474	0.6615	2.3510	-88.0308	-61.6352	-1.6191	-1.6201
0.19	1.6602	850	0.9144	-3.7476	-6.0958	0.6659	2.3482	-87.5153	-61.1479	-1.6336	-1.6346
0.1631	1.7578	900	0.9176	-3.7680	-6.1169	0.6681	2.3489	-87.7256	-61.3513	-1.6457	-1.6466
0.1537	1.8555	950	0.9149	-3.7653	-6.1210	0.6659	2.3558	-87.7673	-61.3244	-1.6467	-1.6476
0.134	1.9531	1000	0.9166	-3.7686	-6.1250	0.6659	2.3564	-87.8066	-61.3577	-1.6465	-1.6475

Framework versions

Transformers 4.40.1
Pytorch 2.0.0+cu117
Datasets 2.19.0
Tokenizers 0.19.1

Downloads last month: 6

Safetensors

Model size

7B params

Tensor type

F16

Model tree for tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT

Base model

mistralai/Mistral-7B-Instruct-v0.2

Finetuned

tsavage68/mistralit2_1000_STEPS_5e7_SFT

Finetuned

(19)

this model