Instructions to use cloudyu/mistral_15B_instruct_v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cloudyu/mistral_15B_instruct_v0.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cloudyu/mistral_15B_instruct_v0.1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cloudyu/mistral_15B_instruct_v0.1")
model = AutoModelForCausalLM.from_pretrained("cloudyu/mistral_15B_instruct_v0.1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use cloudyu/mistral_15B_instruct_v0.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cloudyu/mistral_15B_instruct_v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cloudyu/mistral_15B_instruct_v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cloudyu/mistral_15B_instruct_v0.1

SGLang

How to use cloudyu/mistral_15B_instruct_v0.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cloudyu/mistral_15B_instruct_v0.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cloudyu/mistral_15B_instruct_v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cloudyu/mistral_15B_instruct_v0.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cloudyu/mistral_15B_instruct_v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cloudyu/mistral_15B_instruct_v0.1 with Docker Model Runner:
```
docker model run hf.co/cloudyu/mistral_15B_instruct_v0.1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

 this is a 64 layers model based on mistral architecture 
 sft by vicgalle/alpaca-gpt4.
 template is "{instruction} {inputs} \n {output}"

example output

<s> write me a story about yosemite.  
 Once upon a time, in the heart of California's Sierra Nevada Mountains, lay the magnificent Yosemite National Park. This natural wonderland was home to towering sequoia trees, crystal-clear rivers, and breathtaking waterfalls that cascaded down from cliff faces. The park attracted visitors from all over the world who were enchanted by its beauty and tranquility.

One sunny day, as the sun rose over the mountains, two hikers set out on their adventure through Yosemite. Their names were Jack and Emily, and they had heard countless stories about the park's wonders. They followed the winding trails through forests of oak, pine, and cedar, taking in the sights and sounds of nature around them. As they walked, they came across a small stream, its waters sparkling in the sunlight. They stopped for a moment to rest and take in the serenity of their surroundings.

As they continued their journey, they reached a point where the trail met with a steep cliff face. Before them lay the majestic Yosemite Falls, the highest waterfall in North America. Jack and Emily stood in awe, feeling tiny in comparison to the grandeur before them. They took pictures and soaked in the incredible view, feeling invigorated by the power of nature.

As the sun began to set, Jack and Emily made their way back to their campsite. They sat around the fire, sharing stories and memories of their adventures throughout the park. They felt grateful for the experience and knew that it was one they would never forget.

Yosemite National Park remained a magical place, full of wonder and mystery. It drew people in with its beauty and left them feeling recharged and inspired. And Jack and Emily, they returned year after year, always finding new ways to explore this natural masterpiece.</s>

Downloads last month: 67

Safetensors

Model size

14B params

Tensor type

BF16

Model tree for cloudyu/mistral_15B_instruct_v0.1

Quantizations

1 model