File size: 1,788 Bytes
6e668dc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | # Engine-Core Smoke Tests
Start headless `vllm`:
```bash
source ../vllm/.venv/bin/activate
HF_HUB_OFFLINE=1 \
VLLM_LOGGING_LEVEL=DEBUG \
VLLM_CPU_KVCACHE_SPACE=2 \
VLLM_HOST_IP=127.0.0.1 \
VLLM_LOOPBACK_IP=127.0.0.1 \
python3 -m vllm.entrypoints.cli.main serve Qwen/Qwen3-0.6B \
--headless \
--enable-sleep-mode \
--data-parallel-address 127.0.0.1 \
--data-parallel-rpc-port 62100 \
--data-parallel-size-local 1 \
--max-model-len 512 \
--dtype float16
```
Run the Rust smoke test through the `vllm-engine-core-client` utility interface:
```bash
cargo run -p vllm-engine-core-client --example external_engine_utility_call -- \
--handshake-address tcp://127.0.0.1:62100 \
--host 127.0.0.1
```
If your current engine setup does not support sleep mode, skip the `sleep` / `wake_up` part of the
smoke:
```bash
cargo run -p vllm-engine-core-client --example external_engine_utility_call -- \
--handshake-address tcp://127.0.0.1:62100 \
--host 127.0.0.1 \
--skip-sleep-wake
```
Run the Rust smoke test for sample logprobs decoding through the raw engine-core request path:
```bash
cargo run -p vllm-engine-core-client --example external_engine_logprobs -- \
--handshake-address tcp://127.0.0.1:62100 \
--host 127.0.0.1
```
This smoke requests a small generated-token `logprobs` payload plus prompt logprobs over a much
longer prompt, so it exercises both the inline and aux-frame decode paths against a real engine.
The Rust client decodes those payloads into semantic per-position records rather than exposing the
raw ndarray/tensor wire shape.
IMPORTANT: You must restart `vllm` each time you run the smoke test, as the vLLM engine cannot manage frontend closures and subsequent reconnects. In other words, do not reuse existing `vllm` instances, if any.
|