Instructions to use KnutJaegersberg/webMistral-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KnutJaegersberg/webMistral-7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="KnutJaegersberg/webMistral-7B", trust_remote_code=True)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("KnutJaegersberg/webMistral-7B", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("KnutJaegersberg/webMistral-7B", trust_remote_code=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use KnutJaegersberg/webMistral-7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "KnutJaegersberg/webMistral-7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KnutJaegersberg/webMistral-7B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/KnutJaegersberg/webMistral-7B
- SGLang
How to use KnutJaegersberg/webMistral-7B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "KnutJaegersberg/webMistral-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KnutJaegersberg/webMistral-7B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "KnutJaegersberg/webMistral-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KnutJaegersberg/webMistral-7B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use KnutJaegersberg/webMistral-7B with Docker Model Runner:
docker model run hf.co/KnutJaegersberg/webMistral-7B
Prompt Example:
## Question: What is slowing down our internet speeds?
### Google Search Results: - Are my other devices slowing down my connection? Gadgets with slower internet technology can slow down speeds for all your other devices. Everyone knows the feeling: you’re hard at work and then suddenly the Internet seems to slow down. Why is that? From background programs to cheap routers, here are a few reasons why your Internet might be a concern. When working outside your home, here’s how to keep your information safe on public Wi-Fi. If your internet slows down only when too many other people are online simultaneously, you’re probably using more bandwidth than your plan allows. Use our internet speed test to see if you’re getting the speed advertised by your ISP. If your results are close to your plan speed, consider upgrading. Generally, your modem or router (or both) will create a speed bottleneck if not working properly—the same goes with wireless gateways. If your equipment is too old, it may not support important internet protocols. Equipment damage, such as bad ports or components, can also cause slowdowns. Is your internet suddenly moving slowly? It could be due to an outdated router or a less-than-ideal router location. Your connection issues may need only an easy fix, like upgrading to a mesh network (which also has to be set up in the right spot) or simply restarting your modem and router. But if you've already attempted many of the tried-and-true methods and your internet speeds are still subpar, the issue might be something your internet service provider is intentionally doing: bandwidth throttling.
### Response: There are several factors that can slow down internet speeds. These include having gadgets with slower internet technology, running background programs[2], using more bandwidth than your plan allows[3], equipment damage[4], an outdated router or a less-than-ideal router location[5], and bandwidth throttling by the internet service provider[5].
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 47.08 |
| ARC (25-shot) | 59.04 |
| HellaSwag (10-shot) | 80.89 |
| MMLU (5-shot) | 59.0 |
| TruthfulQA (0-shot) | 39.71 |
| Winogrande (5-shot) | 76.32 |
| GSM8K (5-shot) | 8.87 |
| DROP (3-shot) | 5.75 |
- Downloads last month
- 486