Instructions to use tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT")
model = AutoModelForCausalLM.from_pretrained("tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT

SGLang

How to use tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT with Docker Model Runner:
```
docker model run hf.co/tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT
```

Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT

This model is a fine-tuned version of tsavage68/mistralit2_1000_STEPS_5e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5990
Rewards/chosen: 0.5762
Rewards/rejected: -0.8738
Rewards/accuracies: 0.6505
Rewards/margins: 1.4500
Logps/rejected: -29.4696
Logps/chosen: -21.7510
Logits/rejected: -2.1561
Logits/chosen: -2.1557

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6838	0.0977	50	0.6757	0.0819	0.0446	0.6088	0.0374	-26.4083	-23.3987	-2.3067	-2.3063
0.5869	0.1953	100	0.5936	0.1585	-0.1222	0.6418	0.2808	-26.9643	-23.1432	-2.2711	-2.2707
0.4715	0.2930	150	0.5452	-0.2129	-0.8058	0.6659	0.5930	-29.2430	-24.3812	-2.2397	-2.2393
0.354	0.3906	200	0.5529	1.0155	0.1855	0.6549	0.8300	-25.9386	-20.2868	-2.2199	-2.2195
0.4396	0.4883	250	0.5574	1.1590	0.1518	0.6462	1.0072	-26.0510	-19.8085	-2.2035	-2.2031
0.3274	0.5859	300	0.5545	1.1199	0.0715	0.6593	1.0484	-26.3185	-19.9386	-2.2082	-2.2078
0.4225	0.6836	350	0.5761	0.8487	-0.3483	0.6440	1.1970	-27.7178	-20.8428	-2.1904	-2.1900
0.438	0.7812	400	0.5743	0.8375	-0.4076	0.6505	1.2451	-27.9155	-20.8801	-2.1868	-2.1864
0.4097	0.8789	450	0.5715	0.9972	-0.2262	0.6593	1.2234	-27.3110	-20.3477	-2.1789	-2.1785
0.3681	0.9766	500	0.5530	1.3124	0.1000	0.6637	1.2124	-26.2237	-19.2971	-2.1811	-2.1807
0.2244	1.0742	550	0.5675	1.0929	-0.2118	0.6549	1.3047	-27.2629	-20.0288	-2.1714	-2.1710
0.1844	1.1719	600	0.5865	0.7455	-0.6438	0.6484	1.3894	-28.7029	-21.1865	-2.1633	-2.1629
0.3499	1.2695	650	0.5943	0.6716	-0.7550	0.6484	1.4266	-29.0734	-21.4330	-2.1596	-2.1592
0.2335	1.3672	700	0.5946	0.6222	-0.8092	0.6440	1.4314	-29.2540	-21.5976	-2.1580	-2.1576
0.1899	1.4648	750	0.5962	0.5886	-0.8572	0.6484	1.4459	-29.4143	-21.7096	-2.1567	-2.1563
0.319	1.5625	800	0.5973	0.5755	-0.8764	0.6440	1.4519	-29.4783	-21.7533	-2.1565	-2.1561
0.2466	1.6602	850	0.5971	0.5726	-0.8773	0.6484	1.4499	-29.4812	-21.7631	-2.1562	-2.1558
0.2674	1.7578	900	0.5953	0.5773	-0.8785	0.6462	1.4559	-29.4853	-21.7472	-2.1565	-2.1560
0.2268	1.8555	950	0.5990	0.5769	-0.8744	0.6462	1.4514	-29.4716	-21.7486	-2.1562	-2.1558
0.235	1.9531	1000	0.5990	0.5762	-0.8738	0.6505	1.4500	-29.4696	-21.7510	-2.1561	-2.1557

Framework versions

Transformers 4.40.1
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

Downloads last month: 5

Safetensors

Model size

7B params

Tensor type

F16

Model tree for tsavage68/Mistral2_1000_STEPS_03beta_1e7rate_CDPOSFT

Base model

mistralai/Mistral-7B-Instruct-v0.2

Finetuned

tsavage68/mistralit2_1000_STEPS_5e7_SFT

Finetuned

(19)

this model