Instructions to use matsuo-lab/weblab-10b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use matsuo-lab/weblab-10b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="matsuo-lab/weblab-10b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("matsuo-lab/weblab-10b") model = AutoModelForCausalLM.from_pretrained("matsuo-lab/weblab-10b") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use matsuo-lab/weblab-10b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "matsuo-lab/weblab-10b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "matsuo-lab/weblab-10b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/matsuo-lab/weblab-10b
- SGLang
How to use matsuo-lab/weblab-10b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "matsuo-lab/weblab-10b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "matsuo-lab/weblab-10b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "matsuo-lab/weblab-10b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "matsuo-lab/weblab-10b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use matsuo-lab/weblab-10b with Docker Model Runner:
docker model run hf.co/matsuo-lab/weblab-10b
Commit History
Update README.md 8ff1eaf
Update README.md 043ea3b
Update README.md f72f467
Update special_tokens_map.json 3f76763
Update README.md 4ae27ec
Update README.md 2dd30eb
Update README.md d27148e
Update README.md 0e13cf6
Update README.md c5e01d8
Update README.md a1205ba
Takeshi Kojima commited on
Update README.md 98be71c
Takeshi Kojima commited on
Upload 2 files 61fb8bf
Takeshi Kojima commited on
Upload pytorch_model-00003-of-00003.bin 6af9da1
Takeshi Kojima commited on
Upload 6 files 0eace9c
Takeshi Kojima commited on
Update README.md f3eb622
Takeshi Kojima commited on
Update README.md 6e52648
Takeshi Kojima commited on