Instructions to use CharlesLi/mistral_llama_2_code_math_0_full with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CharlesLi/mistral_llama_2_code_math_0_full with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CharlesLi/mistral_llama_2_code_math_0_full")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CharlesLi/mistral_llama_2_code_math_0_full")
model = AutoModelForCausalLM.from_pretrained("CharlesLi/mistral_llama_2_code_math_0_full")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CharlesLi/mistral_llama_2_code_math_0_full with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CharlesLi/mistral_llama_2_code_math_0_full"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CharlesLi/mistral_llama_2_code_math_0_full",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CharlesLi/mistral_llama_2_code_math_0_full

SGLang

How to use CharlesLi/mistral_llama_2_code_math_0_full with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CharlesLi/mistral_llama_2_code_math_0_full" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CharlesLi/mistral_llama_2_code_math_0_full",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CharlesLi/mistral_llama_2_code_math_0_full" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CharlesLi/mistral_llama_2_code_math_0_full",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CharlesLi/mistral_llama_2_code_math_0_full with Docker Model Runner:
```
docker model run hf.co/CharlesLi/mistral_llama_2_code_math_0_full
```

CharlesLi commited on Jan 19, 2025

Commit

e18dc28

verified ·

1 Parent(s): 4f0f349

Model save

Browse files

Files changed (10) hide show

README.md +1 -0
all_results.json +8 -3
eval_results.json +8 -0
model-00001-of-00003.safetensors +1 -1
model-00002-of-00003.safetensors +1 -1
model-00003-of-00003.safetensors +1 -1
runs/Jan20_00-00-50_dgx-a100-14/events.out.tfevents.1737328598.dgx-a100-14.585960.1 +3 -0
runs/Jan20_00-08-05_dgx-a100-13/events.out.tfevents.1737328636.dgx-a100-13.964584.0 +3 -0
train_results.json +3 -3
training_args.bin +1 -1

README.md CHANGED Viewed

@@ -5,6 +5,7 @@ base_model: mistralai/Mistral-7B-Instruct-v0.1
 tags:
 - trl
 - sft
 - generated_from_trainer
 datasets:
 - generator

 tags:
 - trl
 - sft
+- alignment-handbook
 - generated_from_trainer
 datasets:
 - generator

all_results.json CHANGED Viewed

@@ -1,9 +1,14 @@
 {
     "epoch": 1.0,
     "total_flos": 261724569600.0,
-    "train_loss": 1.163278341293335,
-    "train_runtime": 18.7864,
     "train_samples": 480,
-    "train_samples_per_second": 4.737,
     "train_steps_per_second": 0.16
 }

 {
     "epoch": 1.0,
+    "eval_loss": 0.8714168667793274,
+    "eval_runtime": 0.7342,
+    "eval_samples": 20,
+    "eval_samples_per_second": 4.086,
+    "eval_steps_per_second": 1.362,
     "total_flos": 261724569600.0,
+    "train_loss": 1.163399616877238,
+    "train_runtime": 18.7784,
     "train_samples": 480,
+    "train_samples_per_second": 4.739,
     "train_steps_per_second": 0.16
 }

eval_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "epoch": 1.0,
+    "eval_loss": 0.8714168667793274,
+    "eval_runtime": 0.7342,
+    "eval_samples": 20,
+    "eval_samples_per_second": 4.086,
+    "eval_steps_per_second": 1.362
+}

model-00001-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:53ff7288e44a8ffaba793ecf04f6c5ad03d174de8cffd773fe501da070ded140
 size 4943162336

 version https://git-lfs.github.com/spec/v1
+oid sha256:a07ea007d78bd0d48bd523688c14319944afea100bef7b3200d7bcb62a1c910f
 size 4943162336

model-00002-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cd5dc2c301730b9960042a6d50e325c9eeb56a793d7cb4ebd48232b95018e873
 size 4999819336

 version https://git-lfs.github.com/spec/v1
+oid sha256:1f15f81ace1df79e40f5925242e8fadcf464ba365d4bea49426331e231085a7f
 size 4999819336

model-00003-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fcb6a6d754a1a71fae2dbf276aa7d9e819b61b0afd90f2319164198873c3ae47
 size 4540516344

 version https://git-lfs.github.com/spec/v1
+oid sha256:6f2cc112490fbbd711ae1844f5a9496379520ccd1d7d56a89d5d1fa279204fa9
 size 4540516344

runs/Jan20_00-00-50_dgx-a100-14/events.out.tfevents.1737328598.dgx-a100-14.585960.1 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9c3ae15291a917298a5b181434fb4ba0802ddcc5dfcac5249471ddc9065f0da9
+size 354

runs/Jan20_00-08-05_dgx-a100-13/events.out.tfevents.1737328636.dgx-a100-13.964584.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cdfd77c23e4d955c93071c83b5625265d9a1de986fc5b94a3ac192e4360e19f0
+size 6206

train_results.json CHANGED Viewed

@@ -1,9 +1,9 @@
 {
     "epoch": 1.0,
     "total_flos": 261724569600.0,
-    "train_loss": 1.163278341293335,
-    "train_runtime": 18.7864,
     "train_samples": 480,
-    "train_samples_per_second": 4.737,
     "train_steps_per_second": 0.16
 }

 {
     "epoch": 1.0,
     "total_flos": 261724569600.0,
+    "train_loss": 1.163399616877238,
+    "train_runtime": 18.7784,
     "train_samples": 480,
+    "train_samples_per_second": 4.739,
     "train_steps_per_second": 0.16
 }

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7dd100349c91880750b55a16682715a5d5b1857529c8c500819d7b1a7ed12bc2
 size 7032

 version https://git-lfs.github.com/spec/v1
+oid sha256:9b59feb474c59127d5f77cf70fa81c08363492b0eb2d92f761d9c18239efc01e
 size 7032