Instructions to use RUCAIBox/mvp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RUCAIBox/mvp with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="RUCAIBox/mvp") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("RUCAIBox/mvp", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use RUCAIBox/mvp with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "RUCAIBox/mvp" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RUCAIBox/mvp", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/RUCAIBox/mvp
- SGLang
How to use RUCAIBox/mvp with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "RUCAIBox/mvp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RUCAIBox/mvp", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "RUCAIBox/mvp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RUCAIBox/mvp", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use RUCAIBox/mvp with Docker Model Runner:
docker model run hf.co/RUCAIBox/mvp
Commit History
Update README 333fcb9
Update README bcc6b22
Update README 354bcfd
Update README 880d1f6
Update README bc67018
Update README 2a080b7
Update README d8f26c8
Update README 0e45617
Update README 4b71193
Update README c531557
Update README d4d2d83
Update README 3f5bb21
Update README d614bd7
Update README 56b1ce2
Update README 879cb74
Update vocab 3e88be9
Tianyi Tang commited on
Update vocab a08abbe
Tianyi Tang commited on
Update vocab ab74966
Tianyi Tang commited on
initial commit 1ba330f
initial commit 31c17cc
initial commit 981a6ab
Tianyi Tang commited on