Instructions to use CharlesLi/mistral_llama_2_code_math_4_full with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CharlesLi/mistral_llama_2_code_math_4_full with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CharlesLi/mistral_llama_2_code_math_4_full")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CharlesLi/mistral_llama_2_code_math_4_full")
model = AutoModelForCausalLM.from_pretrained("CharlesLi/mistral_llama_2_code_math_4_full")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use CharlesLi/mistral_llama_2_code_math_4_full with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CharlesLi/mistral_llama_2_code_math_4_full"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CharlesLi/mistral_llama_2_code_math_4_full",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CharlesLi/mistral_llama_2_code_math_4_full

SGLang

How to use CharlesLi/mistral_llama_2_code_math_4_full with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CharlesLi/mistral_llama_2_code_math_4_full" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CharlesLi/mistral_llama_2_code_math_4_full",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CharlesLi/mistral_llama_2_code_math_4_full" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CharlesLi/mistral_llama_2_code_math_4_full",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CharlesLi/mistral_llama_2_code_math_4_full with Docker Model Runner:
```
docker model run hf.co/CharlesLi/mistral_llama_2_code_math_4_full
```

CharlesLi commited on Jan 20, 2025

Commit

7bb756d

verified ·

1 Parent(s): 51cc130

Model save

Browse files

Files changed (10) hide show

README.md +1 -0
all_results.json +9 -4
eval_results.json +8 -0
model-00001-of-00003.safetensors +1 -1
model-00002-of-00003.safetensors +1 -1
model-00003-of-00003.safetensors +1 -1
runs/Jan20_00-57-52_dgx-a100-13/events.out.tfevents.1737331762.dgx-a100-13.1005098.1 +3 -0
runs/Jan20_01-06-27_dgx-a100-14/events.out.tfevents.1737331600.dgx-a100-14.655707.0 +2 -2
train_results.json +4 -4
training_args.bin +1 -1

README.md CHANGED Viewed

@@ -5,6 +5,7 @@ base_model: mistralai/Mistral-7B-Instruct-v0.1
 tags:
 - trl
 - sft
 - generated_from_trainer
 datasets:
 - generator

 tags:
 - trl
 - sft
+- alignment-handbook
 - generated_from_trainer
 datasets:
 - generator

all_results.json CHANGED Viewed

@@ -1,9 +1,14 @@
 {
     "epoch": 1.0,
     "total_flos": 4763387166720.0,
-    "train_loss": 0.6144113825715106,
-    "train_runtime": 264.9731,
     "train_samples": 7980,
-    "train_samples_per_second": 5.54,
-    "train_steps_per_second": 0.174
 }

 {
     "epoch": 1.0,
+    "eval_loss": 0.5740787982940674,
+    "eval_runtime": 0.7569,
+    "eval_samples": 20,
+    "eval_samples_per_second": 5.285,
+    "eval_steps_per_second": 1.321,
     "total_flos": 4763387166720.0,
+    "train_loss": 0.6139289477597112,
+    "train_runtime": 257.2977,
     "train_samples": 7980,
+    "train_samples_per_second": 5.705,
+    "train_steps_per_second": 0.179
 }

eval_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "epoch": 1.0,
+    "eval_loss": 0.5740787982940674,
+    "eval_runtime": 0.7569,
+    "eval_samples": 20,
+    "eval_samples_per_second": 5.285,
+    "eval_steps_per_second": 1.321
+}

model-00001-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5a40261db1b7ba32f05a98b0fb869b2d16a2e57af5606e3d48621954945dd154
 size 4943162336

 version https://git-lfs.github.com/spec/v1
+oid sha256:b9cb749dd61a277098adc6dddecc880eae82181df441523908716b6b635fb01d
 size 4943162336

model-00002-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f1cc43200f8473833f17758bcbf4b6dff0c2ec04ce281256ceaf3cdadd57c46d
 size 4999819336

 version https://git-lfs.github.com/spec/v1
+oid sha256:05b4daee984a6496d68f5e1878c6deba6b6b90d3786a2425ddeae24ebc640a07
 size 4999819336

model-00003-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:df401546eb56b64a05e6d1457a13f0a09add2260daddcb7f3299bb6a54fad43c
 size 4540516344

 version https://git-lfs.github.com/spec/v1
+oid sha256:377f888b1afb8888b3224ff5d6fe33683afb703590f41ee760693c95771dff4e
 size 4540516344

runs/Jan20_00-57-52_dgx-a100-13/events.out.tfevents.1737331762.dgx-a100-13.1005098.1 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a9e83d318eca30f4d199171159cd0ecbad0e1be4813ea302677b1ca74896b732
+size 354

runs/Jan20_01-06-27_dgx-a100-14/events.out.tfevents.1737331600.dgx-a100-14.655707.0 CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8dda58b7d97438b60452afb76c1b12f7826f24ebc9308abb67c0253373207dcd
-size 5858

 version https://git-lfs.github.com/spec/v1
+oid sha256:2ef1f19f44421045bc57a51baeaf73cdcdfc111edcf35c695cc0a7153be1928b
+size 6206

train_results.json CHANGED Viewed

@@ -1,9 +1,9 @@
 {
     "epoch": 1.0,
     "total_flos": 4763387166720.0,
-    "train_loss": 0.6144113825715106,
-    "train_runtime": 264.9731,
     "train_samples": 7980,
-    "train_samples_per_second": 5.54,
-    "train_steps_per_second": 0.174
 }

 {
     "epoch": 1.0,
     "total_flos": 4763387166720.0,
+    "train_loss": 0.6139289477597112,
+    "train_runtime": 257.2977,
     "train_samples": 7980,
+    "train_samples_per_second": 5.705,
+    "train_steps_per_second": 0.179
 }

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e13f48deecd588d69b971adc1201a810b6469443bf5f676437b7901e87e4966b
 size 7032

 version https://git-lfs.github.com/spec/v1
+oid sha256:432ce073d46aa8f991ef4ccf4b07b19bdb86a23939c22ec86368fd3459e48b7f
 size 7032