Instructions to use cloudyu/mistral_9B_instruct_v0.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cloudyu/mistral_9B_instruct_v0.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cloudyu/mistral_9B_instruct_v0.2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cloudyu/mistral_9B_instruct_v0.2")
model = AutoModelForCausalLM.from_pretrained("cloudyu/mistral_9B_instruct_v0.2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use cloudyu/mistral_9B_instruct_v0.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cloudyu/mistral_9B_instruct_v0.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cloudyu/mistral_9B_instruct_v0.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cloudyu/mistral_9B_instruct_v0.2

SGLang

How to use cloudyu/mistral_9B_instruct_v0.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cloudyu/mistral_9B_instruct_v0.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cloudyu/mistral_9B_instruct_v0.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cloudyu/mistral_9B_instruct_v0.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cloudyu/mistral_9B_instruct_v0.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cloudyu/mistral_9B_instruct_v0.2 with Docker Model Runner:
```
docker model run hf.co/cloudyu/mistral_9B_instruct_v0.2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

 this is a 40 layers model based on mistral architecture 
 sft by vicgalle/alpaca-gpt4.
 template alpaca format

example output

<s> write me a story about yosemite.

Once upon a time, in the heart of California's Sierra Nevada mountains, there lay a breathtakingly beautiful place known as Yosemite National Park. This natural wonderland was home to some of the most magnificent geological formations and scenic vistas that could be found anywhere on earth. It was a place where nature had been given free reign to create its masterpieces without any interference from mankind.

Yosemite was a land of towering sequoia trees, cascading waterfalls, crystal-clear streams, and majestic cliffs that seemed to stretch upwards into the heavens themselves. The air was always cool and fresh, filled with the sweet scent of pine and the soft sound of birds chirping in the treetops. Visitors were constantly amazed by the sheer beauty and grandeur of this magical place.

One particular day, as the sun began to rise over the horizon, a young couple, Sarah and Jack, made their way through the ancient forest, hand in hand. They had come to Yosemite for their honeymoon, eager to explore the wonders of this natural paradise together. As they walked deeper into the park, they marveled at the sights around them – the towering granite domes, the sparkling rivers, and the vibrant colors of the wildflowers that carpeted the ground beneath their feet.

Sarah and Jack continued their journey, taking in all the beauty that Yosemite had to offer. They climbed up the steep trails to the top of the mountain overlooks, gazing out at the stunning panoramic views before them. They sat quietly by the side of the river, watching as the fish swam gracefully through the clear waters. And they shared stories and memories, laughing and smiling as they looked into each other’s eyes.

As the days passed, Sarah and Jack grew more and more enamored with Yosemite. They felt like they were part of something greater than themselves – a connection not only to each other but also to the incredible natural world that surrounded them. They left no trace behind, leaving only footprints in the sand and memories in their hearts.

When it was finally time for Sarah and Jack to leave Yosemite, they both knew that they would carry these experiences with them forever.

Downloads last month: 111

Safetensors

Model size

9B params

Tensor type

BF16

Model tree for cloudyu/mistral_9B_instruct_v0.2

Quantizations

1 model