Instructions to use OuteAI/Lite-Mistral-150M-v2-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OuteAI/Lite-Mistral-150M-v2-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="OuteAI/Lite-Mistral-150M-v2-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("OuteAI/Lite-Mistral-150M-v2-Instruct") model = AutoModelForCausalLM.from_pretrained("OuteAI/Lite-Mistral-150M-v2-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use OuteAI/Lite-Mistral-150M-v2-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OuteAI/Lite-Mistral-150M-v2-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OuteAI/Lite-Mistral-150M-v2-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/OuteAI/Lite-Mistral-150M-v2-Instruct
- SGLang
How to use OuteAI/Lite-Mistral-150M-v2-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OuteAI/Lite-Mistral-150M-v2-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OuteAI/Lite-Mistral-150M-v2-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OuteAI/Lite-Mistral-150M-v2-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OuteAI/Lite-Mistral-150M-v2-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use OuteAI/Lite-Mistral-150M-v2-Instruct with Docker Model Runner:
docker model run hf.co/OuteAI/Lite-Mistral-150M-v2-Instruct
Adding code for usage with HuggingFace transformers.
#1
by puettmann - opened
README.md
CHANGED
|
@@ -133,6 +133,28 @@ This model uses a specific chat format for optimal performance.
|
|
| 133 |
[The model's response]</s>
|
| 134 |
```
|
| 135 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
## Risk Disclaimer
|
| 137 |
|
| 138 |
By using this model, you acknowledge that you understand and assume the risks associated with its use. You are solely responsible for ensuring compliance with all applicable laws and regulations. We disclaim any liability for problems arising from the use of this open-source model, including but not limited to direct, indirect, incidental, consequential, or punitive damages. We make no warranties, express or implied, regarding the model's performance, accuracy, or fitness for a particular purpose. Your use of this model is at your own risk, and you agree to hold harmless and indemnify us, our affiliates, and our contributors from any claims, damages, or expenses arising from your use of the model.
|
|
|
|
| 133 |
[The model's response]</s>
|
| 134 |
```
|
| 135 |
|
| 136 |
+
## Usage with HuggingFace transformers
|
| 137 |
+
The model can be used with HuggingFace's `transformers` library:
|
| 138 |
+
```python
|
| 139 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 140 |
+
|
| 141 |
+
model = AutoModelForCausalLM.from_pretrained("OuteAI/Lite-Mistral-150M-v2-Instruct")
|
| 142 |
+
tokenizer = AutoTokenizer.from_pretrained("OuteAI/Lite-Mistral-150M-v2-Instruct")
|
| 143 |
+
|
| 144 |
+
def generate_response(message):
|
| 145 |
+
# Encode the formatted message as input ids
|
| 146 |
+
input_ids = tokenizer.encode(f"<s>user\n{message}</s>", return_tensors="pt")
|
| 147 |
+
output = model.generate(input_ids, max_length=100, pad_token_id=tokenizer.eos_token_id)
|
| 148 |
+
|
| 149 |
+
# Decode the generated output
|
| 150 |
+
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
|
| 151 |
+
|
| 152 |
+
return generated_text
|
| 153 |
+
|
| 154 |
+
message = "What is the capital of Spain?"
|
| 155 |
+
response = generate_response(message)
|
| 156 |
+
```
|
| 157 |
+
|
| 158 |
## Risk Disclaimer
|
| 159 |
|
| 160 |
By using this model, you acknowledge that you understand and assume the risks associated with its use. You are solely responsible for ensuring compliance with all applicable laws and regulations. We disclaim any liability for problems arising from the use of this open-source model, including but not limited to direct, indirect, incidental, consequential, or punitive damages. We make no warranties, express or implied, regarding the model's performance, accuracy, or fitness for a particular purpose. Your use of this model is at your own risk, and you agree to hold harmless and indemnify us, our affiliates, and our contributors from any claims, damages, or expenses arising from your use of the model.
|