File size: 786 Bytes
e8f3b97 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | # Runtime quickstart
## 1. Start Qwen3-8B
```bash
python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3-8B \
--served-model-name qwen3-8b-base \
--dtype bfloat16 \
--max-model-len 2048 \
--gpu-memory-utilization 0.50 \
--max-num-seqs 8
```
## 2. Clone this repo
```bash
git clone https://huggingface.co/koreallmdev/qwen3-8b-korean-coding-router
cd qwen3-8b-korean-coding-router
```
## 3. Run router CLI
```bash
export OPENAI_BASE_URL=http://localhost:8000/v1
export OPENAI_MODEL=qwen3-8b-base
bash scripts/run_qwen3_8b_korean_coding_router_chat.sh
```
## 4. Test
```text
/cat linux 8010 포트가 이미 사용 중일 때 ss -lntp와 ps를 포함해서 확인 순서 알려줘
/cat longform 주제는 코딩이고 4천자 이상 텍스트를 써줘
```
|